Methods and apparatus for audience data collection and analysis in a content delivery network

ABSTRACT

Methods and apparatus for collection and processing of data relating to users of a content-delivery network. In one embodiment, the content delivery network is a cable or satellite or HFCu network, and the apparatus includes an architecture for routinely harvesting, parsing, processing, and storing data relating to the activities of the users (e.g., subscribers) of the network. In one variant, at least portions of the data are anonymized to protect subscriber privacy.

PRIORITY AND RELATED APPLICATIONS

This application is a continuation of and claims priority to co-owned,co-pending U.S. patent application Ser. No. 12/877,062 of the same titlefiled on Sep. 7, 2010 which claims priority to commonly owned U.S.Provisional Patent Application Ser. No. 61/260,346 of the same titlefiled on Nov. 11, 2009, each of which is incorporated herein byreference in its entirety. This application is related to commonly ownedU.S. Provisional Patent Application Ser. No. 61/256,903 filed on Oct.30, 2009 and entitled “METHODS AND APPARATUS FOR PACKETIZED CONTENTDELIVERY OVER A CONTENT DELIVERY NETWORK”, which is incorporated hereinby reference in its entirety.

COPYRIGHT

A portion of the disclosure of this patent document contains materialthat is subject to copyright protection. The copyright owner has noobjection to the facsimile reproduction by anyone of the patent documentor the patent disclosure, as it appears in the Patent and TrademarkOffice patent files or records, but otherwise reserves all copyrightrights whatsoever.

BACKGROUND OF THE INVENTION

1. Field of Invention

The invention relates generally to the field of data manipulation andanalysis. Specifically, in one exemplary aspect, the invention relatesto computer-assisted methods and apparatus for collection andclassification of data regarding an audience in a content deliverynetwork, such as a cable television or satellite network.

2. Description of Related Technology

Content delivery and distribution networks may have a large number ofdisparate users. In many situations, it is desirable that thepreferences and behaviors of these disparate users be known to theoperators of the network (as well as the content sources which generatethe content for distribution over the network). Moreover, in cases wherethe users are subscribers or customers of the delivery network (e.g., asin a cable television, satellite, Hybrid Fiber over Copper (HFCu), orsimilar network), revenue, profit, and subscriber retention/addition arealso critical concerns, since these factors effectively keep the networkoperator (and to some degree content producers) commercially viable.Billing systems and other support systems have accordingly been set upwithin such networks in order to facilitate tracking of customers orsubscribers in terms of subscription level, access privileges, accountstatus (e.g., payments, delinquency), requests for changes in service,and other related functions.

Prior art implementations however, generally lack the ability to rapidlyand accurately track user behavior, preferences, and activity (includinguser interaction with programs or applications), and analyze this datain a timely fashion so as to e.g., allow for “tailored” advertisementinsertion, selection of content for delivery to the users. These priorart systems are also generally entirely reactive; i.e., do notincorporate real-time data to proactively adjust the delivery ofcontent, advertisements, etc. in advance of a user's request orspecified preference.

SUMMARY OF THE INVENTION

The present invention addresses the foregoing needs by providing, interalia, methods and apparatus adapted to collect and analyze audiencedata.

In a first aspect of the invention, a method for the collection andanalysis of data relating to the delivery of content is disclosed. Inone embodiment, the content is delivered via a content delivery network(e.g., cable, satellite, HFCu, etc.), and the method comprises:receiving one or more content-related data elements; monitoring userinteraction with the content to generate usage data records; processingthe usage data records; and transmitting the one or more content-relateddata elements and the processed usage data records to an analysisentity. The analysis entity is configured to analyze the data elementsand usage records, and generate one or more reports therefrom.

In one variant, the one or more reports are subsequently utilized forfuture content delivery decisions.

In a second aspect of the invention, a data acquisition and analysishardware architecture for use in a content delivery network isdisclosed.

In a third aspect of the invention, a data acquisition and analysissoftware architecture for use in a content delivery network isdisclosed.

In a fourth aspect of the invention, a CPE with data acquisition andtransmission capabilities for use with the above-referenced architectureis disclosed.

In a fifth aspect of the invention, a data analysis engine is disclosed.

In a sixth aspect of the invention, a database architecture formaintaining various types of user/content-related data is disclosed.

In a seventh aspect of the invention, a computer-readable apparatus isdisclosed. In one embodiment, the apparatus comprises at least onecomputer program adapted to analyze multi-source data relating to userbehavior and activity.

In an eighth aspect, method for collecting and analyzing of datarelating to delivery of content via a network is disclosed. In oneembodiment, the method comprises: (i) collecting a plurality of useractivity data, the user activity data comprising data relating to usageor interaction by a user with the content; (ii) receiving a plurality ofreference data, the reference data describing the content; (iii)correlating at least a portion of the user activity data to thereference data to create viewing impressions; and (iv) transmitting theviewing impressions to an entity for analysis thereof.

In a ninth aspect, a method for managing data in a operator-managedcontent delivery network is disclosed. In one embodiment, the methodcomprises: (i) receiving a first plurality data, the first plurality ofdata comprising a plurality of fields comprising data relating to asubscriber's interaction with a plurality of content delivered thereto,at least a portion of the first plurality of data being: validated so asto ensure a device from which the first plurality of data is received isassociated with the subscriber; and parsed to individual ones of theplurality of fields; (ii) receiving a second plurality of data, thesecond plurality of data comprising data describing the plurality ofcontent; and (iii) processing the individual ones of the plurality offields to generate a plurality of records correlating the individualones of the plurality of fields to the reference data, the plurality ofrecords correlating the individual ones of the plurality of fields tothe reference data being utilized to describe a period of time in whichthe subscriber is presumed to be utilizing a particular one of theplurality of content.

In a tenth aspect, an apparatus for collection and analysis of datarelating to delivery of content in a network is disclosed. In oneembodiment, the apparatus comprises: first logic configured to collect aplurality of data relating to a plurality of primary and secondarycontent delivered to a plurality of subscriber devices via the network;and second logic configured to collect a plurality of data relating toindividual ones of a plurality of tuning events which occur at theplurality of subscriber devices, the plurality of subscriber devicesbeing associated with one or more of a plurality of subscribers.

In an eleventh aspect, a network apparatus for use in a network isdisclosed, in one embodiment, the network apparatus comprises: aninterface configured to communicate with the network; processingapparatus in communication with the interface; at least one storagedevice in communication with the processing apparatus, the storagedevice comprising a computer program which is configured to, whenexecuted on the processing apparatus: (i) obtain data relating to atleast one of each primary and secondary content elements delivered to aplurality of client apparatus via the network; (ii) collect a pluralityof data relating to individual ones of a plurality of tuning eventswhich occur at the plurality of client apparatus; and (iii) process theobtained data and the collected plurality of data so as to identify oneor more correlations between the tuning events and the delivered primaryand secondary content elements.

These and other features and advantages of the present invention willimmediately be recognized by persons of ordinary skill in the art withreference to the attached drawings and detailed description of exemplaryembodiments as given below.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a functional block diagram illustrating an exemplary HFC cablenetwork configuration useful with the present invention.

FIG. 1a is a functional block diagram illustrating one exemplary HFCcable network headend configuration useful with the present invention.

FIG. 1b is a functional block diagram illustrating one exemplary localservice node configuration useful with the present invention.

FIG. 1c is a functional block diagram illustrating one exemplarybroadcast switched architecture (BSA) network useful with the presentinvention.

FIG. 1d is a functional block diagram illustrating one exemplarypacketized content delivery network architecture useful with the presentinvention.

FIG. 2 is a block diagram illustrating an exemplary overarchingarchitecture for the collection and analysis of data.

FIG. 2a is block diagram detailing the components of the data collectionsystem of FIG. 2.

FIG. 2b is a block diagram detailing a second embodiment of thecomponents of the data collection system of FIG. 2.

FIG. 3 is a logical flow diagram illustrating a method for thecollection of SDV data.

FIG. 4 is a logical flow diagram illustrating a method for collectingsubscriber data.

FIG. 5 is a logical flow diagram illustrating a method for thecollection of VOD data.

FIG. 6 is a logical flow diagram illustrating a method for AS datacollection.

FIG. 7 is a logical flow diagram illustrating a method for advertisementdata collection.

FIG. 8 is a logical flow diagram illustrating a method for EPG datacollection.

FIG. 9 is a logical flow diagram illustrating a method for processingcollected SDV records.

FIG. 10 is a logical flow diagram illustrating a method for managing ASdata records.

FIG. 11 is a logical flow diagram illustrating a method of EPG datarecord management and processing.

FIG. 12 is a logical flow diagram illustrating a task process for themanagement of VOD data records.

FIG. 13a illustrates an exemplary global control table.

FIG. 13b illustrates an exemplary system control table.

FIG. 14 is a logical flow diagram illustrating a task process for themanagement of advertisement data records.

FIG. 15 is a logical flow diagram illustrating a method for managingsubscriber data records.

FIG. 16 is a logical flow diagram illustrating a method for recyclingtuning events.

FIG. 17 is a logical flow diagram illustrating a method for recyclingVOD events.

FIG. 18 is a logical flow diagram illustrating a method of recyclingadvertising events.

FIG. 19 is a logical flow diagram illustrating a method for recyclingtuning filtered events.

FIG. 20 is a block diagram illustrating an architecture for theinteraction between the data collection system and SDA.

FIG. 21 illustrates interaction between a collector and other componentswithin the data collection system.

FIG. 22 is a block diagram illustrating a network architectureimplementation for the data collection and data analysis systems.

FIG. 23 is a block diagram illustrating another exemplary architecturefor the collection and analysis of data.

FIG. 24 is a state diagram illustrating the collected tuner events andstate descriptions used to map viewing impression states.

FIG. 25 is a second state diagram illustrating the collected tunerevents and state descriptions used to map the viewing impressionsstates.

FIG. 26 is a state diagram illustrating the collected VOD events andstate descriptions used to map the VOD state.

FIG. 27 is a matrix of the network elements and the downstream.

FIG. 28 is a block diagram illustrating how an advertisement insertionevent is used to determine viewership when storing advertisementinsertion events.

FIG. 29 is a block diagram illustrating how the platform elements areutilized to deliver interactive applications.

FIG. 30 is a state diagram illustrating various interactive applicationstates and transitions between these states which lead to the generationof interactive application events.

FIG. 31 is a state diagram illustrating various tuner states and avisualization of the valid transitions between these states.

FIG. 32 is a matrix of the valid TUNE OUT events and appropriate tuningevents.

All figures © Copyright 2009-2010 Time Warner Cable, Inc. All rightsreserved.

DETAILED DESCRIPTION OF THE INVENTION

Reference is now made to the drawings, wherein like numerals refer tolike parts throughout.

As used herein, the term “advertisement” refers to (without limitation)advertising, promotions, info-mercials, related segments, or specialfeatures (e.g., the making of “X”, where X is the primary content), aswell as other types of secondary content.

As used herein, the term “application” refers generally to a unit ofexecutable software that implements a certain functionality or theme.The themes of applications vary broadly across any number of disciplinesand functions (such as on-demand content management, e-commercetransactions, brokerage transactions, home entertainment, calculatoretc.), and one application may have more than one theme. The unit ofexecutable software generally runs in a predetermined environment; forexample, the unit could comprise a downloadable Java Xlet™ that runswithin the JavaTV™ environment.

As used herein, the terms “client device” and “end user device” include,but are not limited to, set-top boxes (e.g., DSTBs), personal computers(PCs), and minicomputers, whether desktop, laptop, or otherwise, andmobile devices such as handheld computers, PDAs, personal media devices(PMDs), and smartphones.

As used herein, the term “computer program” or “software” is meant toinclude any sequence or human or machine cognizable steps which performa function. Such program may be rendered in virtually any programminglanguage or environment including, for example, C/C++, Fortran, COBOL,PASCAL, assembly language, markup languages (e.g., HTML, SGML, XML,VoXML), and the like, as well as object-oriented environments such asthe Common Object Request Broker Architecture (CORBA), Java™ (includingJ2ME, Java Beans, etc.), Binary Runtime Environment (e.g., BREW), andthe like.

The terms “Customer Premises Equipment (CPE)” and “host device” refer toany type of electronic equipment located within a customer's or user'spremises and connected to a network. The term “host device” refersgenerally to a terminal device that has access to digital televisioncontent via a satellite, cable, or terrestrial network. The host devicefunctionality may be integrated into a digital television (DTV) set. Theterm “customer premises equipment” (CPE) includes such electronicequipment such as set-top boxes (e.g., DSTBs), televisions, cable modems(CMs), embedded multimedia terminal adapters (eMTAs), whetherstand-alone or integrated with other devices, Digital Video Recorders(DVR), gateway storage devices (Furnace), and ITV Personal Computers.

As used herein, the term “database” refers generally to one or moretangible or virtual data storage locations, which may or may not bephysically co-located with each other or other system components.

As used herein, the term “display” means any type of device adapted todisplay information, including without limitation CRTs, LCDs, TFTs,plasma displays, LEDs, incandescent and fluorescent devices. Displaydevices may also include less dynamic devices such as, for example,printers, e-ink devices, and the like.

As used herein, the term “DVR” (digital video recorder) refers generallyto any type or recording mechanism and/or software environment wherebycontent sent over a network can be recorded and selectively recalled.Such DVR may be dedicated in nature, or part of a non-dedicated ormulti-function system.

As used herein, the term “DOCSIS” refers to any of the existing orplanned variants of the Data Over Cable Services InterfaceSpecification, including for example DOCSIS versions 1.0, 1.1, 2.0 and3.0. DOCSIS (version 1.0) is a standard and protocol for internet accessusing a “digital” cable network.

As used herein, the term “headend” refers generally to a networkedsystem controlled by an operator (e.g., an MSO) that distributesprogramming to MSO clientele using client devices. Such programming mayinclude literally any information source/receiver including, inter alia,free-to-air TV channels, pay TV channels, interactive TV, and theInternet.

As used herein, the terms “Internet” and “internet” are usedinterchangeably to refer to inter-networks including, withoutlimitation, the Internet.

As used herein, the terms “microprocessor” and “digital processor” aremeant generally to include all types of digital processing devicesincluding, without limitation, digital signal processors (DSPs), reducedinstruction set computers (RISC), general-purpose (CISC) processors,microprocessors, gate arrays (e.g., FPGAs), PLDs, reconfigurable computefabrics (RCFs), array processors, secure microprocessors, andapplication-specific integrated circuits (ASICs). Such digitalprocessors may be contained on a single unitary IC die, or distributedacross multiple components.

As used herein, the terms “MSO” or “multiple systems operator” refer toa cable, satellite, or terrestrial network provider havinginfrastructure required to deliver services including programming anddata over those mediums.

As used herein, the terms “network” and “bearer network” refer generallyto any type of telecommunications or data network including, withoutlimitation, hybrid fiber coax (HFC) networks, satellite networks, telconetworks, and data networks (including MANs, WANs, LANs, WLANs,internets, and intranets). Such networks or portions thereof may utilizeany one or more different topologies (e.g., ring, bus, star, loop,etc.), transmission media (e.g., wired/RF cable, RF wireless, millimeterwave, optical, etc.) and/or communications or networking protocols(e.g., SONET, DOCSIS, IEEE Std. 802.3, ATM, X.25, Frame Relay, 3GPP,3GPP2, WAP, SIP, UDP, FTP, RTP/RTCP, H.323, etc.).

As used herein, the term “node” refers without limitation to anylocation, functional entity, or component within a network.

As used herein, the term “QAM” refers to modulation schemes used forsending signals over cable networks. Such modulation scheme might useany constellation level (e.g. QPSK, 16QAM, 64QAM, 256QAM, etc.)depending on details of a cable network. A QAM may also refer to aphysical channel modulated according to the schemes.

As used herein, the term “network interface” refers to any signal, data,or software interface with a component, network or process including,without limitation, those of the Firewire (e.g., FW400, FW800, etc.),USB (e.g., USB2), Ethernet (e.g., 10/100, 10/100/1000 (GigabitEthernet), 10-Gig-E, etc.), MoCA, Serial ATA (e.g., SATA, e-SATA,SATAII), Ultra-ATA/DMA, Coaxsys (e.g., TVnet™), radio frequency tuner(e.g., in-band or OOB, cable modem, etc.), WiFi (802.11a,b,g,n), WiMAX(802.16), PAN (802.15), or IrDA families.

As used herein, the term “storage device” refers to without limitationcomputer hard drives, DVR device, memory, RAID devices or arrays,optical media (e.g., CD-ROMs, Laserdiscs, Blu-Ray, etc.), or any otherdevices or media capable of storing content or other information.

As used herein, the term “user interface” refers to, without limitation,any visual, graphical, tactile, audible, sensory, or other means ofproviding information to and/or receiving information from a user orother entity.

As used herein, the term “wireless” means any wireless signal, data,communication, or other interface including without limitation WiFi(such as IEEE-Std. 802.11 or related standards including 802.11a/b/g/n), Bluetooth, 3G, HSDPA/HSUPA, TDMA, CDMA (e.g., IS-95A, WCDMA,etc.), FHSS, DSSS, GSM, PAN/802.15, WiMAX (802.16), 802.20,narrowband/FDMA, OFDM, PCS/DCS, analog cellular, CDPD, satellitesystems, millimeter wave or microwave systems, acoustic, and infrared(i.e., IrDA).

Overview

In one salient aspect, the present invention discloses methods andapparatus for audience data collection and analysis in a contentdelivery network (such as a cable, satellite or HFCu network). In oneembodiment, the invention provides for the collection of data fromvarious data sources, such as inter alia, a device/user data source, answitched digital video (SDV) data source, a subscriber data source, avideo on-demand (VOD) data source, an application server (AS) datasource, an advertisement data source, and an electronic program guide(EPG) data source. Collected data is transmitted to a data collectionsystem, where the records are collected, processed and used to generatefiles for delivery to a subscriber data analysis (SDA) system, thedelivered data being used for detailed analysis of user preferences,activity, and/or behavior.

Data is collected, parsed (if necessary), transformed and/or managedprior to transmission or reporting to the SDA (and other entities). Oneof the primary objectives of the SDA is to provide a reportinginfrastructure from the collected data that provides usage, namely theSDV tuning events, VOD events and advertisement events. In one variant,reports are generated and transmitted, including: (i) householdviewership, (ii) household commercial/advertisement viewership, and(iii) VOD consumption.

Reference data (e.g., static data) is loaded directly into the SDAthrough the use of a loader entity or through direct insertion. The MSOmay provide facilities to keep internal reference data up to date,however, in some instances, records fail because the data isinconsistent with the subscriber information or the network. To accountfor that, the system advantageously provides a recycling mechanism toreprocess records that could not find matching records for. Theaforementioned collection system may also provide the ability to log anerror if connectivity is lost between elements in the collection system.

Input files, as collected, are in one embodiment archived or stored intoa dedicated directory on the local file system. Additionally, exception(error, filter, and duplicate) files may be stored into a dedicateddirectory on the local file system as well.

A specific interface for retrieving audience measurement events fromsample data sets prior to the transmission to the SDA is also discussedherein. In one variant, household data, channel information, and tuningdata are collected via the interface.

In order to protect subscriber anonymity, customer identifiers (e.g.,MAC address, subscriber account numbers, customer account numbers) may,in one embodiment, be made anonymous after being is correlated withgeneric demographic information prior to its delivery.

Information may also be collected under the present invention from theuse of interactive applications (e.g., “iTV” or similar interactiveapplications). Business methods and systems of implementation of theforegoing are also discussed herein.

DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS

Exemplary embodiments of the audience classification apparatus andmethods of the present invention are now described in detail. Whilethese exemplary embodiments are described in the context of a hybridfiber coax (HFC) cable architecture having a multiple systems operator(MSO), digital networking capability, and plurality of clientdevices/CPE, the general principles and advantages of the invention maybe extended to other types of networks and architectures, whetherbroadband, narrowband, optical or electrical, wired or wireless, contentor data, or otherwise. Hence, the following description is merelyexemplary in nature. For example, the invention may be practiced over afiber-to-the-home (FTTH) or fiber-to-the-curb (FTTC) system, HFCunetworks, or over satellite or millimeter wave-based networks havingtwo-way capabilities.

It will also be appreciated that while described generally in thecontext of a network providing service to a customer or consumer (i.e.,residential) end user domain, the present invention may be readilyadapted to other types of environments including, e.g.,commercial/enterprise, and government/military applications. Myriadother applications are possible.

Also, while certain aspects are described primarily in the context ofthe well known IP or Internet Protocol (described in, inter alia, RFC791 and 2460), it will be appreciated that the present invention mayutilize other types of protocols (and in fact bearer networks to includeother internets and intranets) to implement the described functionality.

Network—

FIG. 1 illustrates a typical content delivery network configuration withwhich the apparatus and methods of the present invention may be used.The various components of the network 100 include (i) one or more dataand application origination points 102; (ii) one or more content sources103, (iii) one or more application distribution servers 104; (iv) one ormore VOD servers 105, and (v) customer premises equipment (CPE) 106. Thedistribution server(s) 104, VOD servers 105 and CPE(s) 106 are connectedvia a bearer (e.g., HFC) network 101. A simple architecture comprisingone of each of the aforementioned components 102, 104, 105, 106 is shownin FIG. 1 for simplicity, although it will be recognized that comparablearchitectures with multiple origination points, distribution servers,VOD servers, and/or CPE devices (as well as different networktopologies) may be utilized consistent with the invention. For example,the headend architecture of FIG. 1a (described in greater detail below)may be used.

The data/application origination point 102 comprises any medium thatallows data and/or applications (such as a VOD-based or “Watch TV”application) to be transferred to a distribution server 104. This caninclude for example a third party data source, application vendorwebsite, CD-ROM, external network interface, mass storage device (e.g.,RAID system), etc. Such transference may be automatic, initiated uponthe occurrence of one or more specified events (such as the receipt of arequest packet or ACK), performed manually, or accomplished in anynumber of other modes readily recognized by those of ordinary skill.

The application distribution server 104 comprises a computer systemwhere such applications can enter the network system. Distributionservers are well known in the networking arts, and accordingly notdescribed further herein.

The VOD server 105 comprises a computer system where on-demand contentcan be received from one or more of the aforementioned data sources 102and enter the network system. These servers may generate the contentlocally, or alternatively act as a gateway or intermediary from adistant source.

The CPE 106 includes any equipment in the “customers' premises” (orother locations, whether local or remote to the distribution server 104)that can be accessed by a distribution server 104.

Referring now to FIG. 1a , one exemplary embodiment of a headendarchitecture useful with the present invention is described. As shown inFIG. 1a , the headend architecture 150 comprises typical headendcomponents and services including billing module 152, subscribermanagement system (SMS) and CPE configuration management module 154,cable-modem termination system (CMTS) and OOB system 156, as well asLAN(s) 158, 160 placing the various components in data communicationwith one another. It will be appreciated that while a bar or bus LANtopology is illustrated, any number of other arrangements as previouslyreferenced (e.g., ring, star, etc.) may be used consistent with theinvention. It will also be appreciated that the headend configurationdepicted in FIG. 1a is high-level, conceptual architecture and that eachMSO may have multiple headends deployed using custom architectures.

The exemplary architecture 150 of FIG. 1a further includes amultiplexer-encrypter-modulator (MEM) 162 coupled to the HFC network 101adapted to process or condition content for transmission over thenetwork. The distribution servers 164 are coupled to the LAN 160, whichprovides access to the MEM 162 and network 101 via one or more fileservers 170. The VoD servers 105 are coupled to the LAN 160 as well,although other architectures may be employed (such as for example wherethe VoD servers are associated with a core switching device such as an802.3z Gigabit Ethernet device). As previously described, information iscarried across multiple channels. Thus, the headend must be adapted toacquire the information for the carried channels from various sources.Typically, the channels being delivered from the headend 150 to the CPE106 (“downstream”) are multiplexed together in the headend as previouslydescribed, and sent to neighborhood hubs (FIG. 1b ) via a variety ofinterposed network components.

It will also be recognized, however, that the multiplexing operation(s)need not necessarily occur at the headend 150 (e.g., in theaforementioned MEM 162). For example, in one variant, at least a portionof the multiplexing is conducted at a BSA switching node or hub (seediscussion of FIG. 1c provided subsequently herein). As yet anotheralternative, a multi-location or multi-stage approach can be used, suchas that described in U.S. patent application Ser. No. 11/048,334,entitled “APPARATUS AND METHODS FOR MULTI-STAGE MULTIPLEXING IN ANETWORK” incorporated herein by reference in its entirety, whichdiscloses inter cilia improved multiplexing apparatus and methods thatallow such systems to dynamically compensate for content (e.g.,advertisements, promotions, or other programs) that is inserted at adownstream network node such as a local hub, as well as “feed back” and“feed forward” mechanisms for transferring information betweenmultiplexing stages.

Content (e.g., audio, video, data, files, etc.) is provided in eachdownstream (in-band) channel associated with the relevant service group.To communicate with the headend or intermediary node (e.g., hub server),the CPE 106 may use the out-of-band (OOB) or DOCSIS channels andassociated protocols. The OCAP 1.0, 2.0, 3.0 (and subsequent)specification provides for exemplary networking protocols bothdownstream and upstream, although the invention is in no way limited tothese approaches.

It will also be recognized that the multiple servers (broadcast, VoD, orotherwise) can be used, and disposed at two or more different locationsif desired, such as being part of different server “farms”. Thesemultiple servers can be used to feed one service group, or alternativelydifferent service groups. In a simple architecture, a single server isused to feed one or more service groups. In another variant, multipleservers located at the same location are used to feed one or moreservice groups. In yet another variant, multiple servers disposed atdifferent location are used to feed one or more service groups.

“Switched” Networks—

FIG. 1c illustrates an exemplary “switched” network architecture alsouseful with the premises gateway apparatus and features of the presentinvention. While a so-called “broadcast switched architecture” or BSAnetwork is illustrated in this exemplary embodiment, it will berecognized that the present invention is in no way limited to sucharchitectures.

Switching architectures allow improved efficiency of bandwidth use forordinary digital broadcast programs. Ideally, the subscriber is unawareof any difference between programs delivered using a switched networkand ordinary streaming broadcast delivery.

FIG. 1c shows the implementation details of one exemplary embodiment ofthis broadcast switched network architecture. Specifically, the headend150 contains switched broadcast control and media path functions 190,192; these element cooperating to control and feed, respectively,downstream or edge switching devices 194 at the hub site which are usedto selectively switch broadcast streams to various service groups. A BSAserver 196 is also disposed at the hub site, and implements functionsrelated to switching and bandwidth conservation (in conjunction with amanagement entity 198 disposed at the headend). An optical transportring 197 is utilized to distribute the dense wave-division multiplexed(DWDM) optical signals to each hub in an efficient fashion.

Co-owned and co-pending U.S. patent application Ser. No. 09/956,688filed Sep. 20, 2001 and entitled “Technique for Effectively ProvidingProgram Material in a Cable Television System”, incorporated herein byreference in its entirety, describes one exemplary broadcast switcheddigital architecture useful with the present invention, although it willbe recognized by those of ordinary skill that other approaches andarchitectures may be substituted.

In addition to “broadcast” content (e.g., video programming), thesystems of FIGS. 1a and 1c (and FIG. 1d , described below) also deliverInternet data services using the Internet protocol (IP), although otherprotocols and transport mechanisms of the type well known in the digitalcommunication art may be substituted. One exemplary delivery paradigmcomprises delivering MPEG-based video content, with the videotransported to user PCs (or IP-based STBs) over the aforementionedDOCSIS channels comprising MPEG (or other video codec such as H.264 orAVC) over IP over MPEG. That is, the higher layer MPEG- or other encodedcontent is encapsulated using an IP protocol, which then utilizes anMPEG packetization of the type well known in the art for delivery overthe RF channels. In this fashion, a parallel delivery mode to the normalbroadcast delivery exists; i.e., delivery of video content both overtraditional downstream QAMs to the tuner of the user's STB or otherreceiver device for viewing on the television, and also as packetized IPdata over the DOCSIS QAMs to the user's PC or other IP-enabled devicevia the user's cable modem.

Referring again to FIG. 1e , the IP packets associated with Internetservices are received by edge switch 194, and forwarded to the cablemodem termination system (CMTS) 199. The CMTS examines the packets, andforwards packets intended for the local network to the edge switch 194.Other packets are discarded or routed to another component.

The edge switch 194 forwards the packets receive from the CMTS 199 tothe QAM modulator 189, which transmits the packets on one or morephysical (QAM-modulated RF) channels to the CPE. The IP packets aretypically transmitted on RF channels that are different that the RFchannels used for the broadcast video and audio programming, althoughthis is not a requirement. The CPE 106 are each configured to monitorthe particular assigned RF channel (such as via a port or socketID/address, or other such mechanism) for IP packets intended for thesubscriber premises/address that they serve.

“Packetized” Networks—

While the foregoing network architectures described herein can (and infact do) carry packetized content (e.g., IP over MPEG for high-speeddata or Internet TV, MPEG2 packet content over QAM for MPTS, etc.), theyare often not optimized for such delivery. Hence, in accordance withanother embodiment of the present invention, a “packet optimized”delivery network is used for carriage of the packet content (e.g., IPTVcontent). FIG. 1d illustrates one exemplary implementation of such anetwork, in the context of a 3GPP IMS (IP Multimedia Subsystem) networkwith common control plane and service delivery platform (SDP), asdescribed in U.S. provisional patent application Ser. No. 61/256,903entitled “METHODS AND APPARATUS FOR PACKETIZED CONTENT DELIVERY OVER ACONTENT DELIVERY NETWORK, previously incorporated herein. Such a networkprovides significant enhancements in terms of, inter alia, commoncontrol of different services, implementation and management of contentdelivery sessions according to unicast or multicast models, etc.;however, it is appreciated that the various features of the presentinvention are in no way limited to any of the foregoing architectures.

I. Audience Measurement and Analysis System—

Referring now to FIG. 2, one embodiment of an architecture for thecollection and analysis of data according to the present invention isshown. As illustrated, data is collected from various headend datasources 201 and transmitted to a data collection system 200. Data mayalso be collected from user CPE 106 via the network 101. Collected datais processed at the system 200. As shown, usage detail records arecollected, processed and used to generate files for delivery to asubscriber data analysis (SDA) system 203 for analysis. In oneembodiment, the collection system 200 includes and is based on an OpenetFusionWorks system as disclosed in Openet “Fusion Works SDL TechnicalReference © 2008, version 2.0,” which is incorporated herein byreference in its entirety, although it will be appreciated that othertypes of systems and implementations may be used with equal success, andthe use of the Fusion Works system is but one possible implementationchoice.

FIG. 2a is a detailed illustration of the various entities of the datacollection system 200. As shown, information is collected from aplurality of data sources 201, such as inter alia, a device/user datasource 205, a broadcast switched or switched digital video (SDV) datasource 207, a subscriber data source 209, a video on-demand (VOD) datasource 211, an application server (AS) data source 213, an advertisementdata source 215, and an electronic program guide (EPG) data source 217.Data is collected by various data collector entities, such as interalia, a data file collector 202, an SDV data collector 204, a subscriberdetails and services data collector 206, a VOD data collector 208, an ASdata collector 210, an advertisement data collector 212, and an EPG datacollector 214. It is appreciated that while only one of each of theabove data collectors is illustrated, data may be collected from anynumber and/or types of data sources by any number and/or type of datacollectors in accordance with the present invention. Moreover,collection functions can be combined or integrated, or divided acrosstwo or more complementary platforms.

Moreover, while certain embodiments are described primarily described interms of switched digital video (SDV) delivery paradigm, other deliveryparadigms may be supported, such as e.g., IP-based delivery (whetherunicast or multicast), the description of SDV being merely for purposesof illustration.

Data files collected from the data file sources) 205 by the data filecollector 202 are directly loaded via a loader 216 to one or moredatabases. The loader 216 generally comprises bulk loader utility usedfor moving data from external files into one or more databases. In oneembodiment, the loader 216 may comprise an SQL*Loader® of the typemanufactured by Oracle® Corporation, although other implementations maybe readily appreciated by those of ordinary skill given the presentdisclosure. As illustrated, the collected data is loaded (via the loader216) to at least one of three databases: (i) a geography database 218,(ii) a demographic profile database 220, and (iii) a deviceconfiguration database 222. It is appreciated that additional databasesfor the storage of data from the data file collector 202 may beprovided, and/or the foregoing databases may be combined or integratedif desired.

Data regarding device and/or user geographic location is stored at thegeography database 218. Data regarding user demographics/psychographicsis stored at the demographic profile database 220. Data regarding deviceconfigurations is stored at the device configuration database 222.

Tuning event information and tuning event types information is collectedat the SDV collection entity 204 for eventual storage at e.g., theviewing impressions database 240. Tuning event data from the SDV servers204 is in the illustrated embodiment collected in real time. Thecollected tuning event data is encoded in plain text. In the illustratedembodiment, data is collected via a transmission control protocol (TCP)port as a TCP/IP stream. In one implementation, the collector comprisesa Cisco™ SDV Collector, software version 1.5, as discussed in Cisco DataSheet entitled “SDV Collector” dated December 2008, and incorporatedherein by reference in its entirety. Exemplary parameters definedspecifically for the collection of SDV data are illustrated in AppendixA of U.S. Provisional Patent Application Ser. No. 61/260,346 of the sametitle filed on Nov. 11, 2009, previously incorporated herein byreference in its entirety.

As noted above, the direct connection protocol is through a TCP/IP link.In one embodiment, there are 5 server ports open per usage type, 15 intotal per SDV. There is also a UDP multicast port available as well. SDVservers 204 deliver records in the exemplary embodiment Log 2 Format asdescribed below. Data is compressed through the TCP/IP stream.

In another embodiment, 8 instances of the collector are given percluster, 1 division to collect in total—7 are active, and 1 stand by.The instances naming convention is as follows:

-   -   <div>_SDV<type>_<method>_<###.###.###.###>        Here, <div> represents the division code, _SDV_(—) represents        the identity of the collector apparatus as a SDV collector. The        symbol <type> refers to the type of collector in this instance a        tuning events collector (TUN). The symbol <method> gives the        method of collection, for example, TCP. Finally,        <##.###.###.###> represents the IP address of the apparatus. A        sample instance name is “B23_SDV_TUN_TCP_10.1.1.1”.

Since data is collected in real time, a file naming conversion is notapplicable. Further, a file name duplicate check is not necessary inthis embodiment, as for real time feeds there is no file nameduplication. Also, a sequence number validation is not necessary.

Data collected from the SDV data source(s) 207 is validated at e.g. thevalidation entity 224. The MAC address of the validated data is thenanonymized at the anonymizer 226. In one embodiment, the MAC address maybe anonymized by inter cilia, the use of a cryptographic hash. In oneembodiment, the techniques for providing anonymity utilizing acryptographic hash described in U.S. patent application Ser. No.11/186,452 filed Jul. 20, 2005 and entitled “Method and Apparatus forBoundary-Based Network Operation”, which is incorporated herein byreference in its entirety, may be utilized in conjunction with thepresent invention. As disclosed therein, the identity of a CPE isanonymized by using a cryptographic hash coupled with an optional“opaque” variable which carries information relating to the CPE of thehash with which it is associated. The hash and opaque variable frustratede-encryption or reverse-engineering of the individual subscriber'sidentity or specific location. Alternative methods of providinganonymization may also be utilized consistent with the presentinvention, including e.g., the anonymization methods discussed hereinbelow.

The enrichment entity 232 is used to enrich the data. Data is enrichedwith anonymous subscriber data (from the subscriber information store234) to add non-identifiable customer information. The enrichment entity232 further adds channel identification, service and/or lineupinformation from the services/channel lineup tables 230 and the servicegroup to hub data store 228. In one embodiment, the services/channellineup tables 230 comprise internal tables of the application server 213(discussed in greater detail below).

Pre-processed data may be stored at the pre-processed data storageentity 236. At the viewing impression converter 238, data is convertedinto viewing impressions. The processed records are inserted, via theinsertion entity 242, into SDV usage tables in a subscriber dataanalysis (SDA) system 203, for analysis. Viewing impressions are storedat the viewing impression storage entity 240. In one embodiment,conversion of data into viewing impression involves the correlation oftune in and tune out events into a single tuning impression record thatconsists of the program viewed, start/end date/time of the viewingevent, tuning event type, tune out type, and anonymized customer/CPEGUIDs.

The SDA 203 generally comprises a database for audience measurementanalytics. In one embodiment, the SDA 203 may comprise an Oracle schema,although other implementations may be used with equal success. Severalreference tables are loaded directly into SDA 203, including inter glia,geography information, device configuration, EPG reference data, andtuning reference data (each of which will be discussed below). In oneembodiment, the aforementioned Oracle SQL*Loader product may be used forloading the SDA 203 database. All collected and processed records thatare stored in the SDA 203 database are available for reporting throughthe analytics engine.

Subscriber details and subscriber billing records may be collected atthe subscriber details and services data collector 206. The data iscollected from one or more SDV data sources 207, parsed, and verified atthe validation entity 224. Subscriber information is pushed to thesystem 200, where a local “collect” is performed. Local collection rulesuse a standard collection library. Collections are preformed at regular,configurable intervals; however, files are delivered at least once a dayin the illustrated embodiment (this period being selectable). A“Start_collect” event script for the collector instance is provided tostart a collection at every configured period. These scripts can also beinvoked manually from the GUI. In addition, a “Stop_collect” script isprovided for each collector instance. This can be executed to halt thecollection process if needed.

An anonymizer 244 is utilized for the anonymization of the subscriberinformation. In one embodiment, as discussed above, a cryptographic hashmay be used for anonymization as discussed in previously incorporatedU.S. patent application Ser. No. 11/186. As noted above, subscriberinformation may be passed to the enrichment entity 232 via a firstmerging entity 246. Subscriber records are also merged by a secondmerging entity 248 into SDA 203 while keeping an internal copy forprocessing. Subscriber data is stored at the subscriber informationstorage entity 250.

In one embodiment, the subscriber details and services data is collectedfrom a Convergys™ Integrated Communications Operations Management System(ICOMS) server. According to this embodiment, subscriber data iscollected with the use of a QBatch collector and COGNOS™ datamodeling/analysis. The QBatch collector is configured for the efficientprocessing of files regardless of the location of those files. TheQBatch collector is configured to retrieve the subscriber files and mayuse parameters such as those listed in the table of Appendix B of thepreviously incorporated U.S. Provisional Patent Application Ser. No.61/260,346.

In another embodiment, there is one (1) instance of the SDV collectorentity 206. The collector 206 instance is named using the followingnaming convention is as follows:

-   -   SUB_DETAIL_<div>, SUB_SERVICES_<div>        In the given naming convention, the <div> symbol represents the        division code. A sample instance name is “SUB_AC_411”.

The file naming convention for reference data files is as follows:

-   -   <Div_D>_<ref_type>_YYYYMMDD.TXT        The <Div_ID> symbol of the given naming convention is the        division identifier code. The <ref type> can be either        “VIDEO_SUB” or “VIDEO_SERVICES”. The YYMMDD represents the date        (year, month day). Lastly, “.TXT” is the file extension. The        following is an example of a COGNOS data filename:        B411_VIDEO_SUB_20090211. TXT.

If the filename being collected has already been collected (within e.g.,a user-configurable period of time), the file is considered a duplicate,and an error message is generated. The file is then stored to aduplicate file directory and processing continues with the next file.The period of time for duplicate check is a configurable parameter. Inone embodiment, for the initial configuration of the system 200 isconfigured to maintain the list for 5 days. In one embodiment, in orderto enable filename duplicate checking, the parameters of Appendix C ofthe previously incorporated U.S. Provisional Patent Application Ser. No.61/260,346 may be implemented.

There is no sequence number in the exemplary file structure; therefore,there is no need to validate the sequence number.

The collector 206 collects data from each billing system that the MSOuses. The collector also collects and stores household packagesinformation, CPE/STB household information, and household information,

Video on-demand (VOD) data collected at the VOD server platform 208 isalso monitored. In one embodiment, the VOD data collector platform 208comprises an Everstream® Video on-Demand Collector (software version2.8.0). The data collector 208 utilizes data manipulation software toprepare the data for use within the system 200. The manipulationsoftware is an Oracle Instance, in one specific embodiment. A scheduledtask may, in one embodiment, be configured to retrieve a data recordusing the exemplary parameters listed in the table of Appendix D of thepreviously incorporated U.S. Provisional Patent Application Ser. No.61/260,346.

In another embodiment, one instance of the database schema used todeliver VOD server usage data is provided; the VOD server 208 deliversusage data through a database connection. Since a single instance isused, there is no need to define a VOD collector 208 instance namingconversion. Furthermore, a file naming convention, a file name duplicatecheck and sequence number validation are not utilized for VOD datacollection in this embodiment.

Data retrieved from the VOD collector 208 is compressed, and isdelivered at a specified periodicity (or based on the occurrence ofevents), such at least once a day. A default data format may be used forthe input of VOD data. In one embodiment, the default data formatcomprises an Oracle format based on the SQL*NET® protocol, althoughother approaches may be used with equal success.

VOD data includes in one variant both usage data 252 and referencetables 254. Records from the usage table 252 are retrieved and validatedat the validation entity 256. The MAC address of the subscriber devicesassociated with the collected data is anonymized as discussed above, viaan anonymization entity 258. Data is enriched with othernon-identifiable subscriber data (such as that collected above withrespect to the subscriber details and services data collector 206). Theprocessed records are inserted, via an insertion entity 262, in the SDA203 for analysis. VOD usage data is stored at a VOD usage database 264.

The reference tables 254 are pushed directly to the SDA 203, via a datamerging entity 266, through DB Links and replicated in the SDA instance203. VOD reference data is stored at a VOD reference database 268. Inone embodiment, most of the reference table 254 data is merely merged tothe SDA 203 via a merging entity 270, and is not processed. However,VOD_LEASE and EQUIPMENT tables may require processing. For example, theEQUIPMENT table may require anonymization of have a MAC address(MAC_ADDRESS field) and the VOD_LEASE table may require removal ofaccount specific information (such as e.g., an ACCOUNT_SID field).

Records provided by the VOD collector 208 are managed as they aredelivered. Management includes archiving, as well as the collection andstorage of VOD asset information, VOD asset metadata, VOD eventinformation, trick play information, and VOD trick type information.

Data collected by the application server (AS) database collector 210 iscollected through the use of files. The collector 210 is configured toretrieve records from the AS data source 213. In one example, the ASdata source 213 retrieves these records using the parameters listed inthe table shown in Appendix E of previously incorporated U.S.Provisional Patent Application Ser. No. 61/260,346. The files basedcollection protocol is FTP over TCP/IP in the illustrated embodiment,although other protocols/transports may be used. These files are notcompressed. Collection is performed at regular configurable intervals;but files are delivered at the specified times (e.g., once a day). Filesthat come from the AS database 210 are read, parsed and validated by avalidation entity 270.

In one embodiment, the application server data source 213 comprises theapplication server discussed in co-owned, co-pending U.S. patentapplication Ser. No. 11/263,015 filed Oct. 2, 2002 and entitled “Networkbased digital information and entertainment storage and deliverysystem”, now published as U.S. Patent Application Publication No.2003/0208767, which claims priority under 35 U.S.C. 119(e) the benefitof U.S. Provisional Application No. 60/377,963 filed on May 3, 2002,each of the foregoing incorporated herein by reference in its entirety.In one variant, the AS 213 is a Mystro™ server device of the typeutilized by the Assignee hereof, although it will be appreciated thatother configurations may be utilized with equal success.

In another embodiment, there are sixteen (16) instances of the AScollector 210. The instances naming convention is as follows:

-   -   <div>_MAS_<type>_<##>        The <div> symbol represents the division code, and the _MAS_(—)        symbol illustrates that the data is obtained from a Mystro        Application server 213. In the given naming convention, the        <type> symbol represents the file type being managed by the        collector 210 (for example, the file type may be a SERVICE file,        a PROGRAM file, or a CHANNEL file, etc.) Lastly, the <##> symbol        is the two-digit identifier. A sample instance name is        “B23_MAS_SERVICE_01”.

The file naming convention on the remote host is as follows:

-   -   <fileid>.txt.gz        In the given example, the <fileid> symbol is a field identifier        which identifies the channels, services or lineups (in lower        case). The .txt.gz. portion identifies the data file in text        compressed using gzip. The following are examples of data        filenames: services.txt.gz, channel.txt.gz.

As will be discussed below, files are collected by the system 200 at adesired frequency or periodicity (e.g., nightly), and transmitted to anSDA. The nightly files contain a data extract from the services, lineupand channel tables.

In the present embodiment, there is no requirement for a filenameduplicate check because the files each have fixed names. Old files aretagged as part of collection and archived. There is no sequence numberin the file, therefore, there is no need to validate the sequencenumber.

If the filename being collected has already been collected within agiven period of time, then the file is considered a duplicate. In caseof duplicate file, an error message is generated and the file is storedto a duplicate file directory and processing continues with the nextfile. The period of time for duplicate check is a configurable parameterwithin the system. In one example, such as for an initial configurationthe system may be configured to maintain the list for 5 days. In onevariant, in order to enable filename duplicate checking, the parametersof Appendix F of previously incorporated U.S. Provisional PatentApplication Ser. No. 61/260,346 may be used. Data collected by the AS210 is either merged with internal enrichment table information at amerging entity 272, or merged to the SDA 203 via an SDA merging entity276. The merged data (via either merging entity 272, 276) is then usedto populate a services table, a channel table and/or a line-up table(collectively referred to as service/channel/line-up tables 274). Theservice/channel/line-up tables 274 are used for correlating the SDVsource indicator to a channel and disclose the available channel lineupand offerings in a division or region.

The AS data collector 210 collects and stores channel map information,lineup information, service information, and service types information.

Advertisement insertion data may be collected at the advertisementservers 212 as well. Data is delivered to the system 200 through adatabase connection to an advertisement data source 215. The data iscollected for example at least once a day, and is not compressed.Collected advertisement data may be delivered through the aforementionedOracle SQL*NET protocol, or another suitable protocol as desired.

In one embodiment, the advertisement servers 212 comprise an OpenTV®Eclipse™ product (software version 4.0.1). A scheduled task isconfigured to retrieve data records using, for example, the parameterslisted in the table of Appendix G of previously incorporated U.S.Provisional Patent Application Ser. No. 61/260,346. Advertisementinsertion data includes usage tables 278 and reference tables 280. Atthe normalization advertisement play entity 282, records from the usagetables 278 are enriched with service information that is obtained fromthe service normalization table 284. Collected records are then merged,via merging entity 286 into the SDA 203 and stored at the processedadvertisement plays database 288. The reference tables 280 are pusheddirectly to the SDA 203 through DB Links via a merging entity 290, andare replicated in the SDA instance 203; no processing of the referencetables 280 is required other than merging the data. Advertisementreference data is then stored at an advertisement reference database291.

In another embodiment, there is one (1) instance of the advertisementdata being delivered. A collector naming convention is not necessary forscheduled tasks. Furthermore, file naming conventions, file nameduplicate checks and sequence number validations are not necessary foradvertisement data collection under this embodiment.

In one implementation, an operator-created reference table (not shown)may be utilized for obtaining service information from advertisementevents that are delivered by the advertisement server 212, and anotherreference table (not shown) for matching service groups to hubs in orderto obtain channel information.

The advertisement server 212 collects and stores inter alia, localadvertisements, advertiser information, information regarding advertisertype, agency information, advertisement spot play, and advertisementstatus code information.

Electronic programming guide (EPG) data collectors 214 providetelevision programming and schedule information. Data from the EPG datacollector 114 is collected from an EPG data source 217 through the useof files. The collector 214 is configured to retrieve records using, forexample, the parameters listed in the table of Appendix H of previouslyincorporated U.S. Provisional Patent Application Ser. No. 61/260,346. Inone embodiment, the EPG data source(s) 217 comprise one or more TribuneMedia Systems™ (TMS) databases. The EPG data is collected, parsed, andvalidated at the validation entity 292 and stored at a staging entity294. The staged data may then be merged into the SDA instance 203 via amerging entity 296. Collected EPG data is stored at the EPG recordsdatabase 297.

In another embodiment there is one instance of the data collector 214delivery. The instances naming convention may be represented as:

-   -   <div>_TMS_<type>        The symbol <div> represents the division code. In the given        example, the symbol _TMB_(—) refers to the Tribune Media System        (TMS) implementation discussed above. The <type> symbol is used        to represent the file type being managed by the collector 114        (for example, PROGRAM, or SCHEDULE, etc.) Hence, a sample        instance name might be “B23_TMS_SCHEDULE”.

The files based collection protocol is FTP over TCP/IP. These files arecompressed using GNU zip or “gzip” compression format, which is a wellknown open software application, although other compression utilitiesmay be used with equal success. The collector 214 in the presentembodiment performs collections at regular configurable intervals; e.g.,with files delivered once a day. The files contain 14 days of “rolling”or moving window data in one embodiment (delivered once a day).

The file naming convention on the remote host may in one embodiment beas follows:

-   -   <tabletype>.txt.gz        In this example, the <table type> symbol describes the table        (such as e.g., a progrec table, skedrec table, statrec table,        progrec table, etc.). The symbol “.txt.gz” indicates the        compression type (e.g., gzip) and identifies the file as a data        file. An exemplary data file name is: progrec.txt.gz. The term        “remote hose” refers to the FTP site from which the collector        (e.g., EPG data collector 214) picks up this file.

The data files are delivered as an encrypted summary file. In oneembodiment, an MD5 cryptographic hash is utilized according to thestandards set forth in Network Working Group standard RFC 1321 entitled“The MD5 Message-Digest Algorithm” and dated April 1992, which isincorporated herein by reference in its entirety. Other hashingalgorithms may be used with equal success as well. The termination fileindicates when a file has been written and closed. The termination filealso indicates when all files are ready to be collected. The file namingconvention on the remote host is as follows:

-   -   <tabletype>.txt.gz.sum        According to this embodiment, the <table type> describes the        table, for example, the table may be a progrec table, a skedrec        table, a statrec table, or a progrec table, etc. The txt.gz.sum        identifies a data file summary with an MD5 hash. An exemplary        encrypted summary file name and/or termination file name is        progrec.txt.gz.

In one embodiment, the termination file is not necessary; the files arenow available for pickup as posted.

As previously noted, if the filename being collected has already beencollected within a configurable period of time, then the file isconsidered a duplicate. However, for Tribune Media Files, all files haveexactly the same name; thus, these files must be renamed uponcollection. In one embodiment, the Tribune Media Files are renamed byadding the collection date (and/or time) to the file name in order tocreate a unique file name. The period of time for duplicate check is aparameter and is configurable. See, for example, Appendix I ofpreviously incorporated U.S. Provisional Patent Application Ser. No.61/260,346.

Since there is no sequence number in the file, there is no need forsequence number validation.

The collected data is maintained in archive for, in one embodiment sevendays, however, shorter and/or longer periods may be utilized, includinge.g., a long-term storage option.

As noted above, the system 200 collects and stores inter alia programinformation, rating code information, program genre information, andgenre information.

In one embodiment, four reports are created every night by a nightlyextraction entity 298. The reports contain information regarding thenumber of subscribers, network, programming information and viewingevents. These reports are generated based in part on data from theprocessed viewing impressions, AS, EPG records, and subscriber data. Thereports are delivered to the National Cable Communications (NCC) entity299 using secure FTP, although other mechanisms may be used.

FIG. 2b illustrates another embodiment of the components of the datacollection system of FIG. 2.

II. Data Acquisition/Collection Methods—

SDV Data Collection—

FIG. 3 illustrates one embodiment of a protocol or message exchangebetween the SDV collector and SDV server for the collection of SDV dataaccording to the invention.

Subscriber Data Collection—

FIG. 4 illustrates one embodiment of a message exchange between asubscriber data server and collector, for collecting subscriber data.

VOD Data Collection—

FIG. 5 illustrates one embodiment of a message exchange between variousentities for the collection of VOD data.

AS Data Collection—

FIG. 6 illustrates one embodiment of a message exchange between variousentities for the collection of VOD data.

Advertisement Data Collection—

FIG. 7 illustrates one embodiment of a message exchange between variousentities for the collection of advertisement data.

EPG Data Collection—

FIG. 8 illustrates one embodiment of a message exchange between variousentities for the collection of EPG data.

III. Parsing—

Parsing in the present context refers to, without limitation, theactivity to decompose each record in its individual fields. During thisprocess, the system 100 dissects each record, and proceeds to manage thedecomposed records and their components according to their file formatand semantics. Since the usage data is being delivered through directdatabase connections, parsing is not required, and is not discussedherein with respect to usage data.

SDV Data Parsing—

In one embodiment, the SDV parsing parameters are described in OpenetFusion Works SDL Technical Reference © 2008, version 2.0, previouslyincorporated herein. The table of Appendix J of previously incorporatedU.S. Provisional Patent Application Ser. No. 61/260,346 shows exemplaryparameters which are defined specifically for SDV collectors.

As noted above, one embodiment of the SDV data collector 104 comprises aCisco SDV Collector, as discussed in Cisco Data Sheet entitled “SDVCollector” previously incorporated herein. The input data formatconsists of LOG 2 records that are transmitted on a TCP/IP link. Anexemplary input data layout is described elsewhere herein.

If a record fails to parse because of improper field formatting or fieldcorruption, it is stored in a generic error repository. The system 100does not stop the collection of SDV records if one of the records fails.

Subscriber Details and Services Data Parsing—

The parsing parameters for the subscriber details and services data maybe of the type defined in the aforementioned Openet Fusion Works SDLTechnical Reference. Appendix K of previously incorporated U.S.Provisional Patent Application Ser. No. 61/260,346 illustrates exemplaryparameters specifically defined for the subscriber details and servicesdata collector 106.

In one embodiment, the data is provided by a custom made script. Theinput format consists of clear text files in which the records havefields that are separated by “pipes” (|) (Pipe Separated Values). Thecollector 106 may manage inter alia subscriber detail extract files.These files contain all the information about subscriber, its accountand equipment in premises. The collector 106 may also manage inter alia,subscriber service extract files; these files contain subscribers byaccount, rather than by account or equipment. The subscriber serviceextract files are used to update the subscriber personal information.

The input data layout is specified in greater detail below.

Application Server Data Parsing—

The parsing of application server data is in one variant of the typedefined in the aforementioned Openet Fusion Works SDL TechnicalReference. The table of Appendix L which can be found in previouslyincorporated U.S. Provisional Patent Application Ser. No. 61/260,346illustrates exemplary parameters specifically defined for theapplication server 110.

The input data format consists of clear text files in which the recordshave fields that are separated by “pipes” (|). The input data layout isspecified in greater detail below.

EPG Data Parsing—

The parsing of EPG data is performed in one variant according to themethods and apparatus disclosed in the aforementioned Openet FusionWorks SDL Technical Reference. Exemplary parameters specifically definedfor the EPG data collector 114 are illustrated in the table of AppendixM of the previously incorporated U.S. Provisional Patent ApplicationSer. No. 61/260,346. In one embodiment, as noted above, the EPG datacollector 114 may comprise a Tribune Media Systems database, Version5.2.

The input data format consists of clear text files in which the recordshave fields that are separated by “pipes” (|). The input data layout isspecified in detail below.

IV. Data Management—

Once data is collected, it is necessary to transform and/or manage thedata prior to transmission or reporting to e.g., the NCC 299 and/or SDA203. For example, the data may be transformed from a first input formatto an output format.

SDV Data Record Management—

Collected SDV tuner records are converted into viewing impressions. Aviewing impression refers in the present context to a quanta of time inwhich a subscriber is presumed to be watching a service. For thepurposes of audience measurement, any viewing impression that is longerthan 1 second is considered a valid impression, although other criteriamay be applied.

FIG. 9 illustrates an exemplary method 900 for processing collected SDVrecords. As shown, per step 902, the thread initialization) is executed.

SDV files are collected and stored as raw records that are then used tocreate viewing impressions. These viewing impressions have all the dataneeded for analysis. Batch start executes before any records isprocessed, and is used to prepare the system before any file can beprocessed. The following functional call may be used at step 902 of themethod for executing the thread init.)

Br_Thread_Init {  Initialize variables #Clean up stage tables }  ©Copyright 2009-2010 Time Warner Cable, Inc. All rights reserved

Next, at step 904, the SDV collector 204 connects to the SDV server 207.If the connection is not successful (step 906), the connection isre-tried. If the connection is successful (step 906), while connected(step 908) a read stream function is executed step 910. When theconnection ends (step 912), a new execute thread in it is begun at step914.

Each SDV records has its DATE, TIME, EVENT, SG_ID, MAC, TUNER_USE,SOURCE, and LUA fields validated. Exemplary validation of the abovefields according to the invention is illustrated in Appendix N ofpreviously incorporated U.S. Provisional Patent Application Ser. No.61/260,346. Invalid records are inserted in the error table (e.g., aBL_SDV_TUNLOG_TCP_ERR table).

Duplication of data in the SDV records must be identified as well. Inembodiment, DATE, TIME, EVENT, SG_ID, MAC, TUNER_USE, SOURCE, and LUAfields are checked for duplicate entries. Duplicate records are insertedinto a duplicate table (e.g., a BL_SDV_TUNLOG_TCP_DUP table).

SDV records that belong to a commercial subscriber are filtered as wellas those viewing impressions that have zero duration.

Records that are not duplicates are processed via inter alia, (i)anonymization of the MAC address, (ii) parsing tuner use data, (iii)computing a tune-out event, (iv) obtaining channel information, (v)calculating the platform, (vi) computing contiguous events, and (vii)determining switched digital events. In one embodiment, theanonymization of the MAC address proceeds according to the followingexemplary pseudocode logic:

Lb_Anonymize_MAC { pm_mac } { # Constant salts SaltNumber_0 = first saltvalue SaltNumber_1 = second salt value DWord_0 = Convert into rightDWord (pm_mac) DWord_1 = ZeroPad(Convert into let DWord (pm_mac) AddSaltNumber_0 to DWord_0. Store result to DWord_0. Carry over anyoverflow bits to DWord_1 Subtract second salt value to DWord_1SaltedDword = Concatenate DWord_1 and DWord_0 Anonymized Mac = ApplyMD5(SaltedDWord) return String(Anonymized Mac) }  © Copyright 2009-2010Time Warner Cable, Inc. All rights reservedIn one embodiment, the parsing of tuner use data occurs according to thefollowing logic:

Lb_Parse_Tuner_Use {pm_tuneruse} { Create an empty list. Splitpm_tuneruse in bits being the Least Significant Bit 0 And the mostsignificant bit 7. If bit 0 is 1 then, add “IS_RECORDING” to list. Ifbit 2 is 1 then add “IS_PPV” to list. Case if the combination of bits 3to 5 is: 00 Add “IS_BACKGROUND” 01 Add “IS_MAIN” 02 Add “IS_PIP” EndCase If bit 7 is 1 then add “IS_FAILED” Return list }  © Copyright2009-2010 Time Warner Cable, Inc. All rights reservedThe tune-out events may be computed as follows:

Lb_Compute_Tune_Out {pm_event pm_tuner_use} { If pm_tuner_use isIS_FAILED then Return a tune out type of 4, Error If pm_event is 1 orpm_event is 2 Return a Tune out type of 0, Channel Change If pm_event is8 or pm_event is 6 then Return a tune out type of 2, Inactive Ifpm_event is 3 or pm_event is 4 or pm_event is 5 then Return a tune outtype of 1, Zombie Else Return a tune out type of 2, Inactive }  ©Copyright 2009-2010 Time Warner Cable, Inc. All rights reservedChannel information may be obtained by the following exemplaryfunctional call:

Lb_ObtainChannel {pm_source, pm_source_group} { Get the list of hubsfrom the Source Group to Hub translation table. Use the hub to obtainthe lineup id from BL_MASLINEUP Use the pm_source and lineup id toobtain the service id from BL_MAS_SERV Return Service Id. }  © Copyright2009-2010 Time Warner Cable, Inc. All rights reservedIn another embodiment, the platform calculation occurs according to thefollowing logic:

Lb_Calculate_Platform {pm_cpe_guid} { Search in hash to see if thispm_cpe_guid has been processed. If it has been processed then, obtainthe platform type. If tuner id is > 1 and the platform type is MDN, thenUpdate hash as ODN. Return ODN. Else Return hashed type. Else  If tunerid is >1 then platform is ODN  Else  Platform is MDN  End If  Storedetermined platform type in hash. End If }  © Copyright 2009-2010 TimeWarner Cable, Inc. All rights reservedContiguous events may be computed, in one embodiment, according to thefollowing functional call:

Lb_Compute_Contiguous {pm_cpe_guid, pm_tuner_id, pm_date, pm_time,pm_lua } { Get previous timestamps for last date use hash using thepm_cpe_guid and pm_tuner_id If previous date is within the number ofseconds described in sp_contiguousTreshold of lua then Store currentdate and time in hash Return true, is contiguous Else Store current dateand time in hash Return false, is not contiguous End If }  © Copyright2009-2010 Time Warner Cable, Inc. All rights reserved

Switched digital (SDV) events may be determined by e.g.:

Br_SDVEVent { User Defined Event } { Create Database Map Foreach fieldin User Defined Event If the field is the SOURCE then If User DefinedEvent(SOURCE) has a “-“ # looks like 1000-1 Split the value in SOURCEinto the source id and index. Store the source id and index in thedatabase map. End if ElseIf field=”MAC_ADDRESS” #Mac Address isdelivered as ############-## Extract the MAC address from the parsedrecord. Anonymize the MAC address by calling LB_Anonymize_MAC Store theanonymized MAC address into the database map. Store the index, the valueat the right of the MAC address In the tuner id field. End If Store thecollected value in the anonymized MAC Next Get subscriber data from theBL_SUB_DETL table by using the anonymized MAC address. If the subscriberdata was not found store the content of the array inBL_SDV_TUNLOG_TCP_UGD and finish Else Use the LUA field as the start ofthe event. Use the DATE_TIME as the end of the tuning event. Calculateduration using the LUA and DATE_TIME fields If Duration is less or equalthan the duration in the or sp_filteringThreashold or thesp_filterCommercial is 1 and record belongs to a commercial subscriberThen Store the content of the array in BL_SDV_TUNLOG_TCP_FLR Else Callthe Lb_Parse_Tuner_Use Routine and use the list To populate if it isRecording, PIP or PPV or Error. Using the Service Group and Source, CallLb_ObtainChannel To obtain the channel that will be populated theSERVICE_ID Retrieve any previous source id and service group that wasstored previously call Lb_ObtainChannel to obtain the channel that willbe populated in PREVIOUS_SERVICE_ID Store current source id and servicegroup as the new previous source id for later retrieval. Computetune_out type by calling Lb_Compute_Tune_Out_type Compute platform typewith the number of tuner id's collected. Compute if the event iscontiguous If is_contiguous is false and TUNE_OUT_TYPE is Channel Changeand platform is MDN Then Subtract the amount seconds described insp_mdnTimeAdjust to TOTAL_VIEW_TIME Subtract the amount of secondsdescribed in sp_mdnTimeAdjust to END_TIME End If Store the content ofthe array in BL_SDV_TUNLOG_TCP_STG. End If End If }  © Copyright2009-2010 Time Warner Cable, Inc. All rights reserved

Batch end executes after all records are processed. The batch endprocess is used to prepare the system before any file can be processed.An exemplary functional call for executing the batch end process is:

Br_ThreadEnd { } { Store all records still in memory intoBL_SDV_TUNLOG_TCP_STG }  © Copyright 2009-2010 Time Warner Cable, Inc.All rights reserved

Appendix O of previously incorporated U.S. Provisional PatentApplication Ser. No. 61/260,346 illustrates exemplaryUDE-to-BL_SDV_TUNLOG_TCP_STG mapping; i.e., the data table structure ofthe anonymized and enriched tuning event record described above. Asshown, several local system-specific codes in the raw tuning eventrecord are converted into a normalized view of the event, complete withchannel/network information, tune-out types, etc. The normalized view ofthe event may then be combined/processed with other enriched tuningevent records.

Database tables are used to match the service group number to a givenHUB. This information is then used to determine the lineup in the ASlineup table. The AS lineup table is represented as e.g., a BL_SG_HUBtable. The AS lineup table is populated and maintained manually into thedatabase. Exemplary AS lineup table definitions consistent with thepresent invention may be defined by the table of Appendix P ofpreviously incorporated U.S. Provisional Patent Application Ser. No.61/260,346.

AS Data Record Management—

Records collected from the AS data source 213 are managed utilizing acollection entity 210. In one embodiment (noted above), the collector210 comprises a QBatch collector of the type previously described.

FIG. 10 illustrates an exemplary method for managing AS data records.The AS collector 210 collects line-up files, channel files, and servicesfiles. AS files from the AS 213 are collected via the AS collector 210and stored in a stage table. The staged records may used to populate anAS table in the system 200 and in the SDA 203.

Per step 1002 of the method, the batch start executes before any recordsare processed. The batch start process is used to prepare the systembefore any file can be processed.

Next, for each parsed record (step 1004) the records are validated step1006. If the validation is not successful (step 1008), the next parsedrecord is examined at step 1020. Per step 1022 the batch end process isreached. If the validation is successful (step 1008), a filtering logicis utilized (step 1010).

If the record is filtered (step 1012), the process continues to step1020 where the next parsed record is evaluated. If the record is notfiltered (step 1012), it is determined whether the record is unique(step 1014). If the record is not unique, a duplicate logic is used atstep 1016; and, if the record is unique a non-duplicate logic is used atstep 1018.

Once the duplicate (1016) or non-duplicate (1018) logic is completed,the next parsed record is examined (step 1020) and the batch end processis started (step 1022).

Lineup Files—

“Lineup” files collected at the AS collector 210 are considered to be acomparatively slow-moving dimension. As records are modified orinserted, the system 200 creates records with an activation andexpiration date. This approach ensures that historical references tothese records are maintained.

In one embodiment, the following functional call may be utilized toexecute the batch start process for lineup files:

Br_BatchStart { Initialize Variables #Clean up stage tables TruncateBL_STG_MAS_LNUP table }  © Copyright 2009-2010 Time Warner Cable, Inc.All rights reservedThe collected records in the illustrated embodiment each have theirREGION_NAME, DIVISION_NAME, DIVISION_CODE, and LINEUP_ID fieldsvalidated. Invalid records are inserted in an error table (e.g.,BL_STG_MAS_LNUP_ERR table). Exemplary parameters for AS records areillustrated in Appendix Q of previously incorporated U.S. ProvisionalPatent Application Ser. No. 61/260,346.

The REGION_NAME, DIVISION_CODE, and LINEUP_ID fields in records from aline-up file are checked for duplication and must be unique. Duplicaterecords are inserted in a duplicate table (e.g., BL_STG_MAS_LNUP_DUPtable).

In one embodiment, the line-up records are not filtered.

Records that are not duplicates are used for processing using thefollowing logic:

Br_NonDuplicateEvent { User Defined Event } { #Create Database Map MapUser Defined Event to an array according to the field translation tableResult Insert translated array into BL_STG_MAS_LNUP }  © Copyright2009-2010 Time Warner Cable, Inc. All rights reservedAn exemplary UDE-to-BL_STG_MAS_LNUP mapping is shown in Appendix R ofpreviously incorporated U.S. Provisional Patent Application Ser. No.61/260,346. Appendix R thereof illustrates the MAS source data used toidentify the channel lineup ID for a particular Division network hub asan initial step in identifying the actual network/channel identified inthe raw tuning event record.

A batch end process is executed after all records are processed, and isused to prepare the system before any file can be processed. In oneembodiment, the following functional call is used to execute the batchend process:

Br_BatchEnd { } { Flush any outstanding records into BL_STG_MAS_LNUP#Merge all rows from stage into final Line-Up table #Records must beupdated if they do exist Get a list of lineup records that are inBL_STG_MAS_LNUP not in BL_MASLINEUP. Insert those records intoBL_MASLINEUP with an Activation Date of Today and an expiration day of3999. For all the records that are already in BL_MASLINEUP. If itchanged, update the expiration date to today. Insert a new entry withthe activation day of today and an expiration Date of 3999 Next For Geta list of lineup records that are not in SDA.BL_MASLINEUP. Insert thoserecords into SDA.BL_MASLINEUP with an Activation Date of Today and anexpiration day of 3999. For all the records that are already inSDA.BL_MASLINEUP. If it changed, update the expiration date to today.Insert a new entry with the activation day of today and an expirationDate of 3999 Next For }  © Copyright 2009-2010 Time Warner Cable, Inc.All rights reserved

An exemplary line-up table (e.g., BL_STG_MAS_LNUP) to active line-uptable (e.g., BL_MASLINEUP) mapping for local storage (e.g., at thesystem 200) and SDA 203 is shown in Appendix S of previouslyincorporated U.S. Provisional Patent Application Ser. No. 61/260,346.

Channel Files—

Channel files from the AS are collected and stored in a stage table. Thestaged channel records are then used to populate the AS table of thedata collection system 200 and at the SDA 203. Channel files are alsoconsidered to be a slow-moving dimension. As channel records aremodified or inserted, the system 200 creates channel records with anactivation and expiration date. This approach again ensures thathistorical references to these channel records are kept.

Batch start executes before any channel record is processed and it isused to prepare the system before any file can be processed. In oneembodiment, the batch start process is represented by the followinglogic:

Br_BatchStart { Initialize Variables #Clean up stage tables TruncateBL_STG_MAS_CHAN table }  © Copyright 2009-2010 Time Warner Cable, Inc.All rights reserved

Each of the channel records has its REGION_NAME, DIVISION_NAME,DIVISION_CODE, LINEUP_ID, SERVICE_ID, and DISPLAY_CHANNEL fieldsvalidated. Exemplary validation criteria are given in Appendix T ofpreviously incorporated U.S. Provisional Patent Application Ser. No.61/260,346.

Invalid channel data records are placed in an error table (e.g.,BL_MASCHANNEL_ERR table).

Duplicate channel records are determined by looking for uniqueness inthe REGION_NAME, DIVISION_CODE, LINEUP_ID, SERVICE_ID, andDISPLAY_CHANNEL fields. Duplicate channel records are inserted in aduplicate table (such as e.g., BL_MASCHANNEL_DUP table).

As above with line-up records, channel records are not filtered.

Channel records that are not duplicates are next processed. In oneembodiment, the following logic is used for processing the AS channelrecords.

Br_NonDuplicateEvent { User Defined Event } { #Create Database Map MapUser Defined Event to an array according to the field translation tableInsert translated array into BL_STG_MAS_CHAN }  © Copyright 2009-2010Time Warner Cable, Inc. All rights reserved

An exemplary UDE-to-BL_STG_MAS_CHAN mapping is illustrated in Appendix Uof previously incorporated U.S. Provisional Patent Application Ser. No.61/260,346.

As shown, the MAS reference data allows the collection system 200 to mapfrom a division channel lineup to a specific network/channel based onservice ID.

A batch end process is executed after all the records have beenprocessed. The batch end process is used to prepare the system beforeany file can be processed. The batch end process may have the functionalcall listed below.

Br_BatchEnd { } { Flush any outstanding records into BL_STG_MAS_CHAN#Merge all rows from stage into final Line-Up table #Records must beupdated if they do exist Get a list of lineup records that are inBL_STG_MAS_CHAN not in BL_MASCHANNEL. Insert those records intoBL_MASCHANNEL with an Activation Date of Today and an expiration day of3999. For all the records that are already in BL_MASCHANNEL. If itchanged, update the expiration date to today. Insert a new entry withthe activation day of today and an expiration Date of 3999 Next For Geta list of lineup records that are not in SDA.BL_MASCHANNEL. Insert thoserecords into SDA.BL_MASCHANNEL with an Activation Date of Today and anexpiration day of 3999. For all the records that are already inSDA.BL_MASCHANNEL. If it changed, update the expiration date to today.Insert a new entry with the activation day of today and an expirationDate of 3999 Next For  }  © Copyright 2009-2010 Time Warner Cable, Inc.All rights reservedServices Files—

Services files from the AS 213 are collected and stored in a stagetable. The staged service records are used to populate the AS table inthe system 200 and SDA instances 203. Services files are also consideredto be a slow-moving dimension. As service records are modified orinserted, the system 200 creates service records with an activation andexpiration day. This ensures that historical references to these recordsare kept.

The batch start executes before any record is processed and it is usedto prepare the system before any file can be processed.

Br_BatchStart { Initialize Variables #Clean up stage tables TruncateBL_STG_MAS_SERV table }  © Copyright 2009-2010 Time Warner Cable, Inc.All rights reserved

Each of the service files have their REGION_NAME, DIVISION_NAME,DIVISION_CODE, LINEUP_ID, SERVICE_ID, SERVICE_TYPE, GUIDE_ID, HD_SD,SD_SOURCE_ID, HD_SOURCE_ID, and SDV fields validated. Invalid servicefiles are inserted in an error table (e.g., BL_MAS_SERV_ERR table).Exemplary validation logic is shown in Appendix V of previouslyincorporated U.S. Provisional Patent Application Ser. No. 61/260,346.

The service files are also checked for uniqueness with respect to theREGION_NAME, DIVISION_CODE, and SERVICE_ID fields. Duplicate records areinserted into a duplicate table (e.g., BL_STG_MAS_SERV_DUP table).

The service records are not filtered. However, all the service recordsdelivered from the AS 213 are processed. In one embodiment, records thatare not duplicates are used for processing using the following logic:

Br_NonDuplicateEvent { User Defined Event } { #Create Database Map MapUser Defined Event to an array according to the field translation table.Insert translated array into BL_STG_MAS_SERV }  © Copyright 2009-2010Time Warner Cable, Inc. All rights reserved

Exemplary UDE-to-BL_STG_MAS_SERV mapping is illustrated in the table ofAppendix W of previously incorporated U.S. Provisional PatentApplication Ser. No. 61/260,346. Appendix W represents the last piece ofthe puzzle that allows SDA to map the raw tuning event record source IDto the actual channel/network being viewed.

The batch end process executes after all record are processed. The batchend process is used to prepare the system before any file can beprocessed.

Br_BatchEnd { } { Flush any outstanding records into BL_MASSERVICE_STG#Merge all rows from stage into final Line-Up table #Records must beupdated if they do exist Get a list of lineup records that are inBL_MASSERVICE_STG not in BL_MASSERVICE. Insert those records intoBL_MASSERVICE with an Activation Date of Today and an expiration day of3999. For all the records that are already in BL_MASSERVICE. If itchanged, update the expiration date to today. Insert a new entry withthe activation day of today and an expiration Date of 3999 Next For Geta list of lineup records that are not in SDA.BL_MASSERVICE. Insert thoserecords into SDA.BL_MASSERVICE with an Activation Date of Today and anexpiration day of 3999. For all the records that are already inSDA.BL_MASERVICE. If it changed, update the expiration date to today.Insert a new entry with the activation day of today and an expirationDate of 3999 Next For  }  © Copyright 2009-2010 Time Warner Cable, Inc.All rights reserved

Exemplary mapping of the BL_MASSERVICE_STG to the BL_MAS_SERV isillustrated in the table of Appendix X of previously incorporated U.S.Provisional Patent Application Ser. No. 61/260,346. This table maps allof the active MAS source data together into a table that is used toenrich raw tuning event record source IDs into normalizednetworks/channels.

EPG Data Record Management—

Records collected from the EPG data source 217 are managed utilizing acollection entity 214. In one embodiment (noted above), the EPG datasource 217 comprises one or more Tribune Media Systems servers. Thecollected EPG data files are, in one embodiment, handled with a QBatchcollector. The collected EPG data files include scheduling records andprogramming records.

An exemplary method of EPG data record management and processing isillustrated in FIG. 11. It is noted that, the logic within thevalidation engine for processing the EPG data differs from that of theprocessing of AS data (discussed elsewhere herein). The logic foridentifying duplicates is also different for the EPG data versus the ASdata (also discussed elsewhere herein). Per step 1102 of the method, thebatch start executes before any records are processed.

Next, for each parsed record (step 1104) the records are validated step1106. If the validation is not successful (step 1108), the next parsedrecord is examined at step 1120. Per step 1122 the batch end process isreached. If the validation is successful (step 1108), a filtering logicis utilized (step 1110).

If the record is filtered (step 1112), the process continues to step1120 where the next parsed record is evaluated. If the record is notfiltered (step 1112), it is determined whether the record is unique(step 1114). If the record is not unique, a duplicate logic is used atstep 1016; and, if the record is unique a non-duplicate logic is used atstep 1118.

Once the duplicate (1116) or non-duplicate (1118) logic is completed,the next parsed record is examined (step 1120) and the batch end processis started (step 1122).

Schedule Records—

The schedule files are collected and stored in a stage area. This actionensures that any records collected in error are handled properly. Afterall schedule records have been collected, they are merged into a table;a copy of the table is held at the system 200 and at the SDA instance203. In one embodiment, an Oracle merge function ensures that newschedule records are inserted and existing ones updated in both tables.The schedule records are rotated by 14 days; hence, the schedule recordson the oldest day remain in the reporting instance for historicalpurposes.

The batch start process is used to prepare the system before any filecan be processed. In one embodiment, the batch start process uses thefollowing logic:

Br_BatchStart { Initialize Variables #Clean up stage tables TruncateBL_STG_TMS_SCHD table }  © Copyright 2009-2010 Time Warner Cable, Inc.All rights reservedEach of the schedule records of the collected EPG data has itsTF_STATION_NUM, TF_DATABASE_KEY, TF_AIR_DATE, TF_AIR_TIME, TF_DURATION,TF_SEX_RATING, TF_LANGUAGE_RATING, TF_DIALOG_RATING, and TF_FV_RATINGfields validated. Invalid schedule records are inserted into an errortable (such as e.g., BL_STG_TMS_SCHD_ERR table).

The TF_STATION_NUM, TF_DATABASE_KEY, TF_AIR_DATE, TF_AIR_TIME, andTF_DURATION fields in the schedule records must be unique. Duplicaterecords are inserted into a duplicate table (e.g., BL_STG_TMS_SCHD_DUPtable).

Schedule records are not filtered. Records that are not duplicates areused for processing using the following logic:

Br_NonDuplicateEvent { User Defined Event } { #Create Database Map MapUser Defined Event to an array according to the field translation tableInsert translated array into BL_STG_TMS_SCHD }  © Copyright 2009-2010Time Warner Cable, Inc. All rights reserved

Exemplary mapping of the UDE to the BL_STG_TMS_SCHD is illustrated inthe table of Appendix Y of previously incorporated U.S. ProvisionalPatent Application Ser. No. 61/260,346. Similar to the various MASsource data files, the TMS schedule files are ingested for use inidentifying the specific programs being viewed per the SDV tuningrecords.

Program Records—

The program files are collected and stored in a stage area. This actionensures that any records collected in error are handled properly. Afterall program records have been collected, they are merged into a table; acopy of the table is held at the system 200 and at the SDA instance 203.In one embodiment, an Oracle merge function ensures that new programrecords are inserted and existing ones updated in both tables. In onevariant, the program records are rotated by 14 days, hence, the programrecords on the oldest day remain in the reporting instance forhistorical purposes.

The batch start process is used to prepare the system before any filecan be processed. In one embodiment, the batch start process uses thefollowing logic:

Br_BatchStart { Initialize Variables #Clean up stage tables TruncateBL_STG_TMS_PROG table }  © Copyright 2009-2010 Time Warner Cable, Inc.All rights reserved

Each programming record has its TF_DATABASE_KEY and TF_TITLE fieldsvalidated. Invalid records are inserted into an error table (e.g.,BL_STG_TMS_PROG_ERR). Exemplary validation logic for programming recordsis given at Appendix Z of previously incorporated U.S. ProvisionalPatent Application Ser. No. 61/260,346.

The programming records are also checked for duplicates at e.g., theirTF_DATABASE_KEY and TF_TITLE fields. Duplicate records are inserted intoa duplicate table (e.g., BL_STG_TMS_PROG_DUP table).

The programming records are not filtered. Programming records that arenot duplicates are then used for processing using the following logic:

Br_NonDuplicateEvent { User Defined Event } { #Create Database Map MapUser Defined Event to an array according to the field translation tableInsert translated array into BL_STG_TMS_PROG }  © Copyright 2009-2010Time Warner Cable, Inc. All rights reserved

Exemplary UDE-to-BL_STG_TMS_PROG mapping is illustrated in Appendix AAof previously incorporated U.S. Provisional Patent Application Ser. No.61/260,346. The table of Appendix AA is the destination for the TMSprogram source data, which includes, inter cilia the program title,genre, rating, actors, credits, episode, etc.

Once all of the records have been processed, a batch end processexecutes and is used to prepare the system before any file can beprocessed. In one embodiment, the batch end process utilizes thefollowing functional call:

Br_BatchEnd { } { Flush any outstanding records into BL_STG_TMS_PROG#Merge all rows from stage into final Schedule table #Records must beupdated if they do exist Perform an Oracle Merge from BL_STG_TMS_PROG toBL_TMS_PROG Perform an Oracle MERGE from BL_STG_TMS_PROG to theBL_TMS_PROG in SDA via DBLINK }  © Copyright 2009-2010 Time WarnerCable, Inc. All rights reserved

Exemplary BL_STG_TMS_PROG-to-BL_TMS_PROG mapping is illustrated inAppendix AB of previously incorporated U.S. Provisional PatentApplication Ser. No. 61/260,346. Appendix AB thereof illustrates thedata staging process for the TMS source data. in one embodiment, theactual source data tables are not used for the enrichment process, butrather a “copied over” version of merged records are used.

VOD Data Record Management—

Video On Demand (VOD) usage data is processed in one embodiment in theaforementioned Oracle environment as a PL/SQL program. The system 200provides a chronological-like facility which allows for the execution ofevents at timed intervals.

FIG. 12 illustrates an exemplary task process for the management of VODdata records. Per step 1202 a run timed process is executed. At step1204 the scheduled task logic is executed.

Data for VOD is delivered through a direct database connection. Thesystem 200 uses a scheduled task to retrieve records. There are twotypes of records that are delivered by the VOD collector 208 of theillustrated embodiment: (i) reference records and (ii) usage records.

Reference records are records that are used to enhance usage records,such as asset titles and genre. Reference records are collected directlyfrom the VOD data source 211 and merged, inserted or updated, in the SDAinstance 203.

Usage records are records that are used to measure VOD usage, such asVOD leases or streams. Usage records are collected by the VOD collector208 and inserted into the SDA instance 203. Every record that iscollected is considered new usage.

The VOD records are validated at the MAC_ADDRESS, ASSET_SID, ASSET_ID,ASSET_COMPONENT_NBR, VIEWING_SECS, VOD_LEASE_ID, CALENDAR_DATE, andSTART_TIME fields. An exemplary validation table for the above fields isgiven at Appendix AC of previously incorporated U.S. Provisional PatentApplication Ser. No. 61/260,346. Invalid records are inserted in theerror table (e.g., BL_VOD_ERR table).

The VOD records are also checked for duplicates at the MAC_ADDRESS,ASSET_SID, ASSET_ID, ASSET_COMPONENT_NBR, VOD_LEASE_ID, CALENDAR_DATE,and START_TIME fields. Duplicate records are inserted into a duplicatetable (e.g., BL_VOD_DUP table).

Commercial subscribers are filtered as part of processing. Filteredrecords are stored in a filtered record table (e.g., BL_VOD_FLR). In oneembodiment, a parameter is provided that is used to control whethercommercial records are filtered or not (discussed below).

In one variant, records are extracted from VOD instances from the fieldsand tables listed in the table of Appendix AD of previously incorporatedU.S. Provisional Patent Application Ser. No. 61/260,346. The recordsthat are collected from the database schema are processed in PL/SQL thatis managed by a scheduled task. The process includes anonymization ofthe MAC address, processing usage tables, processing reference tables,and processing VOD data.

In one embodiment, the anonymization uses the following logic:

Lb_Anonymize { pm_mac } { # Constant salts SaltNumber_0 = first saltvalue SaltNumber_1 = second salt value DWord_0 = Convert into rightDWord (pm_mac) DWord_1 = ZeroPad(Convert into let DWord (pm_mac) AddSaltNumber_0 to DWord_0. Store result to DWord_0. Carry over anyoverflow bits to DWord_1 Subtract second salt value to DWord_1SaltedDword = Concatenate DWord_1 and DWord_0 Anonymized Mac = ApplyMD5(SaltedDWord) return String(Anonymized Mac) }  © Copyright 2009-2010Time Warner Cable, Inc. All rights reservedThe usage tables may be processed using the following exemplary logic:

Lb_ProcessUsageTables { } { recordset = Select records for that day thatare needed to be processed Foreach record in recordset IfValidate(record) = Failure then  Insert record into BL_VOD_ERR Else Maprecord into an array according to the data Translation table Fetchsubscriber information by using the MAC Address Anonymized MAC AddressIf subscriber information is not found then Insert array into BL_VOD_UGDElse If subscriber is commercial and the sp_filterCommercial is on.Insert array into BL_VOD_FLR Else Result = Insert array into BL_VOD EndIf If Result =Failure THEN If it failed because of a duplicate errorthen Result = Insert array into BL_VOD_DUP Else Rollback  Causeexception End If Next  Commit Insert records from the BL_ADS_STG to theSDA table BL_TNG via  DBLINK. }  © Copyright 2009-2010 Time WarnerCable, Inc. All rights reservedThe reference tables may be processed using the following exemplarylogic:

ProcessReferenceTables { } { Foreach table If reference table is notequipment -- Done as dynamic oracle statement -- to be done onVOD_ASSET_TITLE, VOD_ASSET, ORGANIZATION -- LOCATION,VOD_ASSET_TITLE_GENRE, VOD_ASSET_TITLE_CATEGORY -- VOD_ASSET_COMPPerform an Oracle MERGE statement from the delivery schema To the SDAinstance via DBLINK. Else -- For equipment, a stage table calledBL_EQUIPMENT_STG -- will be used to ensure that the MAC_ADDRESS is --Anonymous Insert into the BL_LEASE_STG the result of Insert into theBL_LEASE_STG the result of Select VOD_LEASE_SID LEASE_START LEASE_ENDLOCATION_SID Anonymize (EQUIPMENT.MAC_ADDRESS) VOD_ASSET_COMP_NBRASSET_SID EQUIPMENT_SID CALENDAR_SID PREVIEW_FLAG TIME_SID VIEWING_SECSPAUSE_VIEWING_SECS RW_VIEWING_SECS FF_VIEWING_SECS NBR_OF_PAUSESNBR_OF_RW NBR_OF_FF FROM VOD_LEASE, EQUIPMENT WHEREVOD_LEASE.EQUIPMENT_SID=EQUIPMENT.EQUIPMENT_SID; Note that the AccountSID is not selected as part of the VOD LEASE selection, removing it fromthe analytics layer.  Insert records records from BL_VOD into the SDAinstance. Next }  © Copyright 2009-2010 Time Warner Cable, Inc. Allrights reservedProcessing VOD data may utilize the following functional call:

VODProcess { } { #VOD Table List is the list of all the tables that willbe used #to create the usage Check the status of all the Video On Demandtables If the tables are ready to be collected then ProcessUsageTablesProcessReferenceTables Propagate all staged tables into SDA Else Waituntil the next time to collect the records. End If }  © Copyright2009-2010 Time Warner Cable, Inc. All rights reserved

Exemplary mapping to the BL_VOD table is illustrated in Appendix AE ofpreviously incorporated U.S. Provisional Patent Application Ser. No.61/260,346. This process addresses the anonymization of the incoming VoDstream/usage data with Customer GUID and CPE GUID, Exemplary mappingfrom BL_VOD to the SDA.BL_VOD is illustrated in Appendix AF (illustratesthe anonymization of the VoD asset data); and exemplary mapping fromVOD_LEASE to the SDA. VOD_LEASE is illustrated in Appendix AG(illustrates the anonymization of the VoD lease data) of previouslyincorporated U.S. Provisional Patent Application Ser. No. 61/260,346.All other reference tables are copied directly from schema to arespective schema in the SDA 203 according to the database definitiondescribed elsewhere herein.

At least one MSO entity is responsible for advertising the availabilityof records with the use of a global control table. The global controltable lists the status of the tables being loaded into the databaseschema. The system 200 uses this table to indicate when the tables areready to be processed. An exemplary embodiment of a global control table1300 is shown in FIG. 13 a.

The system 200 iterates through the global control table 1300 lookingfor tables that were completed but that were not processed by the system200. If any tables are found, the system 200 transfers the contentaccording to the logic described above. After the transfer is completed,the system 200 creates an entry in a system control table. An exemplarysystem control table 1350 is illustrated in FIG. 13b . The systemcontrol table 1350 is used to track the transfers and to ensure thatincomplete tables or already process tables have been transferred.

As illustrated in FIG. 13a , the system 200 inserts an entry with thetable name, date when the transfer was completed, its disposition andthe number of rows that were transferred. The system control table 200is used to ensure that only those tables that were not transferred areprocessed. This is accomplished by obtaining the oldest date on thistable, and looking for complete transfers in the global control table1300 that have an older date than the one in the system control table1350.

Advertisement Data Record Management—

Advertisement data is processed in one embodiment in the aforementionedOracle environment as a PL/SQL program. The system 200 provides achronological-like facility which allows for the execution of events attimed intervals.

FIG. 14 illustrates an exemplary task process for the management ofadvertisement data records. Per step 1402, a run-timed process isexecuted. At step 1404, the scheduled task logic is executed.

Advertisement data is delivered through a direct database connection.The system 200 uses a scheduled task to manage the process. There aretwo types of records that are delivered by the advertisement datacollector 212 of the present embodiment: (i) reference records, and (ii)usage records.

References records are records that are used to enhance usage records,such as copy or invoice information. Reference records are collecteddirectly from the advertisement data source 215 and merged, inserted orupdated, in the SDA instance 203.

Usage records are records that are used to measure advertisement usage,such as the spots that were broadcasted. Usage records are collected bythe advertisement data collector 212 and inserted into the SDA instance203. Every record that is collected is considered new usage.

Each advertisement records has its SPOT, AIR_DATE, SPOT_START_TIME,COPYLENGTH, CAMPAIGN_ID, CUSTOMER_ID, AGENCY_ID, COPY_ID, NETWORKNAME,SYSCODE, and RETAILUNITCODE fields validated. Exemplary validation logicfor the advertisement data is illustrated in Appendix AH of previouslyincorporated U.S. Provisional Patent Application Ser. No. 61/260,346.Invalid records are inserted in the error table (e.g., BL_ADS_ERRtable).

Advertisement records are also checked for duplicates. In oneembodiment, duplicates are identified by checking the AIR_DATE,SPOT_START_TIME, COPYLENGTH, NETWORKNAME, SYSCODE, and RETAILUNITCODEfields. Duplicate records are inserted into a duplicate table (e.g., theBL_ADS_DUP table). None of the advertisement data records is filtered.

Records are, in one embodiment, extracted from advertisement events fromthe fields and tables listed in the table of Appendix AI of previouslyincorporated U.S. Provisional Patent Application Ser. No. 61/260,346.The records that are collected from the database schema are processed inPL/SQL that is managed by a scheduled task. The process includesprocessing usage tables, processing reference tables, and processingadvertisement data.

In one embodiment, the processing of usage tables is effected via thefollowing logic:

Lb_ProcessUsageTables { } {  recordset = Select records for that daythat are needed to be processed from SPOTS, INVOICE, INVOICE_DETAIL,AGENCY, COPY, COPY_GROUP and SECONDARYCOMMODITY Foreach record inrecordset If record fails validation then Insert record into BL_ADS_ERRElse  Map record into an array according to the field  translation tableEnd If Use the Headend, Network and Syscode to retrieve The service IDfrom the BL_SVC_ALIGNMENT table. If no records are found then UseNetwork and the “Default” headend to retrieve the service_id frombl_svc_alignment. If no records are found then  Insert the processedarray  into the BL_ADS_UGD End If End If If only one record is found,then Insert the processed array into BL_ADS_STG If two records arefound, one for HD and one for SD then Insert two records into theprocessed array into BL_ADS_STG For the SD and HD. Else Insert theprocessed array into the BL_ADS_UGD End If Next  Insert records from theBL_ADS_STG to the SDA table BL_TNG via  DBLINK. }  © Copyright 2009-2010Time Warner Cable, Inc. All rights reserved

Processing the reference tables may, in one embodiment, occur accordingto the following functional call:

ProcessReferenceTables { } { Foreach table If reference table is notequipment -- Done as dynamic oracle statement -- to be done on REGION,AGENCY, RETAILUNIT, COPY_GROUP -- SECONDARY_COMMODITY, ADVERTISER,CONTRACT, COPY, HEADEND -- COPY, NETWORK, SPOTS, INVOICE_DETAIL, PROGRAMPerform an Oracle MERGE statement from the delivery schema To the SDAinstance via DBLINK. Next }  © Copyright 2009-2010 Time Warner Cable,Inc. All rights reservedProcessing the advertisement data may occur according to the followinglogic:

ADSProcess { } { #Ad Plays Table List is the list of all the tables thatwill be used #to create the usage Check the status of all the Video OnDemand tables If the tables are ready to be collected thenProcessUsageTables ProcessReferenceTables Propagate all staged tablesinto SDA Else Wait until the next time to collect the records. End If  } © Copyright 2009-2010 Time Warner Cable, Inc. All rights reserved

Exemplary mapping to the BL_ADS_STG is given in Appendix AJ ofpreviously incorporated U.S. Provisional Patent Application Ser. No.61/260,346. As illustrated, the incoming advertisement reference data isenriched to make it usable for the identification of advertisementplayouts in a given ad zone. Exemplary mapping from BL_TNG_STG to BL_TNGis shown in Appendix AK of previously incorporated U.S. ProvisionalPatent Application Ser. No. 61/260,346. Appendix AK thereof illustratesan exemplary staging table of substantially similar reference data tothat of previously referenced Appendix AJ, which is actually used toenrich the advertisement events received. All other reference tables arecopied directly from schema to a respective schema in SDA 203 accordingto the database definition described elsewhere herein.

Control tables, similar to those discussed above with respect to VODdata management, may be utilized in the management of advertisementdata. Two database tables are used in the illustrated embodiment tomatch the raw records coming from the advertisement data source 214 tothe rest of the data available in SDA 203; one of them may be activelyused for enrichment. These tables are: (i) the geographic alignmenttable (BL_GEO_ALIGNMENT), and (ii) the service alignment table(BL_SVC_ALIGNMENT).

An exemplary geographic alignment table is illustrated in Appendix AL ofpreviously incorporated U.S. Provisional Patent Application Ser. No.61/260,346. The geographic alignment table and the service alignmenttable are built from reference data obtained by the MSO. The geographicalignment table is used to identify the geographic territory that isreached by a given commercial broadcast. For example, when advertisementplay-out occurs, the only geographical information that is available(such as from the Open TV solution) is the spot identificationinformation and the retail unit code.

An exemplary service alignment table is illustrated in Appendix AM ofpreviously incorporated U.S. Provisional Patent Application Ser. No.61/260,346. The service alignment table contains the service levelinformation from an advertisement play-out. The service alignment tableis used to correlate the advertisement play-out to a video service. Inone example, the only information available (such as from the Open TVsolution) are the spot and network names. The network names aredifferent from the ones in services; hence the need for the servicealignment table.

For the purposes of mediation, only the service alignment(BL_SVC_ALIGNMENT) is of interest. During the mediation process,advertisement events are enriched with the data from the servicealignment table to obtain service identification information. Thispermits having a common identifier between the viewing events and theadvertisement play-outs. Since multiple services can be associated to anetwork code, the retail unit code is used to discern which service isthe one of interest to a given advertisement play, because localorigination programs share advertisement network codes for differentservices.

Subscriber Data Record Management—

Records collected from the subscriber data source 209 are managedutilizing a collection entity 206. Subscriber files are, in oneembodiment (noted above), handled with a QBatch collector.

FIG. 15 illustrates an exemplary method for managing subscriber datarecords. The subscriber data collector 206 collects detail files andservices files. Subscriber data files from the subscriber data source209 are collected via the subscriber data collector 206 and stored in astage table. The staged records may used to populate subscriber datatables in the system 200, and in the SDA 203. In one embodiment, thesubscriber data source 209 comprises a Convergys™ ICOMS sever whichdelivers a complete set or “dump” of subscriber data records. The system200 handles additions, updates and removals. To preserve historicalperspective or state, these records may optionally contain connectand/or disconnect dates.

Per step 1502 of the method, the batch start executes before any recordsare processed. The batch start process is used to prepare the systembefore any file can be processed.

Next, for each parsed record (step 1504) the records are validated step1506. If the validation is not successful (step 1508), the next parsedrecord is examined at step 1520. Per step 1522 the batch end process isreached. If the validation is successful (step 1508), a filtering logicis utilized (step 1510).

If the record is filtered (step 1512), the process continues to step1520 where the next parsed record is evaluated. If the record is notfiltered (step 1512), it is determined whether the record is unique(step 1514). If the record is not unique, a duplicate logic is used atstep 1516; and, if the record is unique a non-duplicate logic is used atstep 1518.

Once the duplicate (1516) or non-duplicate (1518) logic is completed,the next parsed record is examined (step 1520) and the batch end processis started (step 1522).

Schedule and Subscriber Records—

A batch start process executes before any schedule record is processedand it is used to prepare the system before any file can be processed.In one embodiment, the batch start process comprises the followinglogic:

Br_BatchStart { Initialize Variables #Clean up stage tables TruncateBL_STG_SUB_DETL table  Truncate BL_SUB_DETLS_CUR table  Insert recordsfrom BL_SUB_DETLS into  BL_SUB_DETLS_CUR }  © Copyright 2009-2010 TimeWarner Cable, Inc. All rights reservedEach of the schedule records has its own fields validated (e.g.,SUBSCRIBER_ZIPCODE, DIVISION, DIVISION_ID, SUBSCRIBER_ACCOUNT, SITE_ID,and SUBSCRIBER_CONNECT_DATE fields). Invalid records are inserted in anerror table (e.g., BL_SUB_DETLS_ERR). Exemplary validation logic isgiven in Appendix AN of previously incorporated U.S. Provisional PatentApplication Ser. No. 61/260,346.

The schedule records are also checked for uniqueness in the DIVISION_ID,SUBSCRIBER_ACCOUNT, and SUBSCRIBER_CONNECT_DATE fields. Duplicateentries are inserted in the duplicate (BL_SUB_DETLS_DUP) table.

The schedule records are not filtered. Records that are not duplicatesare used for processing using the following exemplary logic:

Lb_Anonymize_GUID {Subscriber Account, Division Code} { ConcatenateSubscriber Account and Division Code giving UID. Using its hexadecimalrepresentation, divide UID in DWORDS Store Dwords in an Array, LeastSignificant in 0, Most Significant in N Loop through the Array: If Dwordbelong to an even subscript: add a third salt value and Carry anyoverflow bits to the next entry Else subtract fourth salt value, Borrowfrom next entry if needed End Loop Convert Dword Array into String fromMost Significant to Least Significant (From N to 0) Perform MD5 Hash onthe Converted String Convert UID into ASCII representation ReturnConverted UID } Br_NonDuplicateEvent { User Defined Event } { #CreateDatabase Map Anonymize the SUBSCRIBER_ACCOUNT field by callingLb_Anonymize_GUID. Map User Defined Event to an array according to thefield translation table Insert translated array into BL_STG_SUB_DETL } © Copyright 2009-2010 Time Warner Cable, Inc. All rights reservedExemplary UDE-to-BL_STG_SUB_DETL mapping is shown in Appendix AO ofpreviously incorporated U.S. Provisional Patent Application Ser. No.61/260,346.

Batch end executes after all records are processed and it is used toprepare the system before any file can be processed. In one embodiment,the batch end process comprises the following functional call:

Br_BatchEnd { } { Flush any outstanding records into BL_STG_SUB_DETL#Merge all rows from stage into final Schedule table # TMS Tables willonly go to SDA #Records must be updated if they do exist Get a list ofsubscriber accounts that were not in the current load by ComparingBL_SUB_DETLS_CUR and BL_STG_SUB_DETL Iterate through that list andupdate the DISCONNECT_TIME field to SYSDATE In BL_SUB_DETLS Perform anOracle Merge from BL_STG_SUB_DETL to BL_SUB_DETLS Perform an OracleMerge from BL_STG_SUB_DETL to BL_SUB_DETLS in SDA via DBLINK }  ©Copyright 2009-2010 Time Warner Cable, Inc. All rights reserved

Exemplary BL_STG_SUB_DETL-to-BL_STG_SUB_DETL mapping is illustrated inAppendix AP of previously incorporated U.S. Provisional PatentApplication Ser. No. 61/260,346. The table of Appendix AP addresses thelocation, line of business, and connect/disconnect details associatedwith each incoming subscriber record. This information will be used asthe basis for the anonymized subscriber information feed, as well asreference data for event record enrichment with subscriber details.

Service Records—

A batch start process executes before any service record is processedand it is used to prepare the system before any file can be processed.In one embodiment, the batch start process comprises the followinglogic:

Br_BatchStart { Initialize Variables #Clean up stage tables TruncateBL_STG_SUB_BILL table  Truncate BL_SUB_BILL_CUR table  Insert recordsfrom BL_SUB_BILL into BL_SUB_BILL_CUR }  © Copyright 2009-2010 TimeWarner Cable, Inc. All rights reserved

Each service record has its SITE_ID, DIVISION, SUBSCRIBER_ACCOUNT,SERVICE_CATEGORY, SITE_ID, SERVICE_CODE, and EQUIPMENT_CONNECT_DATEfields validated. Invalid records are inserted in an error table (e.g.,BL_SUB_BILL_ERR table). Exemplary validation logic is given at AppendixAQ of previously incorporated U.S. Provisional Patent Application Ser.No. 61/260,346.

The following fields in the service records are unique: MAC_ADDRESS,DIVISION, SUBSCRIBER_ACCOUNT, SERVICE_CODE, and EQUIPMENT_CONNECT_DATE.Duplicate records are inserted in the duplicate table (e.g.,BL_SUB_BILL_DUP).

The subscriber service records are not filtered. Records that are notduplicates are processed using the following exemplary logic:

Lb_Anonymize_MAC { pm_mac } { # Constant salts SaltNumber_0 = first saltvalue SaltNumber_1 = second salt value DWord_0 = Convert into rightDWord (pm_mac) DWord_1 = ZeroPad(Convert into let DWord (pm_mac) AddSaltNumber_0 to DWord_0. Store result to DWord_0. Carry over anyoverflow bits to DWord_1 Subtract second salt value to DWord_1SaltedDword = Concatenate DWord_1 and DWord_0 Anonymized Mac = ApplyMD5(SaltedDWord) return String(Anonymized Mac) } Lb_Anonymize_GUID{Subscriber Account, Division Code} { Concatenate Subscriber Account andDivision Code giving UID. Using its hexadecimal representation, divideUID in DWORDS Store Dwords in an Array, Least Significant in 0, MostSignificant in N Loop through the Array: If Dword belong to an evensubscript: add third salt value and Carry any overflow bits to the nextentry Else subtract fourth salt value, Borrow from next entry if neededEnd Loop Convert Dword Array into String from Most Significant to LeastSignificant (From N to 0) Perform MD5 Hash on the Converted StringConvert UID into ASCII representation } Br_NonDuplicateEvent { UserDefined Event } { Anonymize the MAC Address field by callingLb_Anonymize_MAC. Anonymize the SUBSCRIBER_ACCOUNT field by callingLb_Anonymize_GUID Map User Defined Event to an array according to thefield translation table Insert translated array into BL_STG_SUB_DETL } © Copyright 2009-2010 Time Warner Cable, Inc. All rights reserved

Exemplary UDE-to-BL_STG_SUB_BILL mapping is illustrated in Appendix ARof previously incorporated U.S. Provisional Patent Application Ser. No.61/260,346. Appendix AR represents the services on each subscriberaccount. This information is used to link the MAC address from the eventrecord to the subscriber and to provide additional details regarding thesubscriber services profile against the enriched usage data.

A batch end process executes after all service records are processed andit is used to prepare the system before any file can be processed. Inone embodiment, the batch end process uses the following logic:

Br_BatchEnd { } { Flush any outstanding records into BL_STG_SUB_BILL#Merge all rows from stage into final Schedule table # TMS Tables willonly go to SDA #Records must be updated if they do exist Get a list ofsubscriber accounts that were not in the current load by ComparingBL_SUB_BILL_CUR and BL_STG_SUB_BILL Iterate through that list and updatethe DISCONNECT_TIME field to SYSDATE In BL_SUB_BILL Perform an OracleMerge from BL_STG_SUB_BILL to BL_SUB_BILL Perform an Oracle Merge fromBL_STG_SUB_BILL to BL_SUB_BILL in SDA via DBLINK }  © Copyright2009-2010 Time Warner Cable, Inc. All rights reserved

Exemplary BL_STG_SUB_BILL-to-BL_SUB_BILL mapping is illustrated inAppendix AS of previously incorporated U.S. Provisional PatentApplication Ser. No. 61/260,346. Appendix AS is one embodiment of astaging table of subscriber services data that may be used to enrichevent records and create the anonymized subscriber data files.

V. Data Delivery—

For data select billing, the system 200 creates downstream feeds foroperation and billing support systems, and file transfer of data betweenthe network and rating instances of the system.

The term “NCC Files” is used herein to collectively refer to “universe”data files, household data files, station data files, and tuning datafiles, which are delivered to e.g., the National Cable Companies (NCC).The system 200 reads directly from the SDA 203 database. The outputformat of the NCC files is one embodiment clear text (in ASCII) usingpipes (|) to separate the fields. All files use the same encoding withdifferent data layouts. The universe data file collects records thatbelonged to all active subscribers, even partially during the period ofinterest.

The NCC files are given names according to the following namingconvention:

-   -   <division>_<year>_<month>.dat.        In the naming convention, the <division> symbol represents the        division name; the <year> symbol represents the creation year in        one embodiment, the year is a four digits number (YYYY). The        <month> symbol is the creation month, in one embodiment, the        creation month is given as two digits, zero padded (MM). The        .dat file extension is a literal. An exemplary field description        table for the NCC records is illustrated in Appendix AT of        previously incorporated U.S. Provisional Patent Application Ser.        No. 61/260,346. Appendix AU thereof illustrates a sample NCC        file and source for data contained therein.

The household data files are records that belonged to all activesubscribers, even partially during this period. The household data filesare named according to the following convention:

-   -   HOUSEHOLD_<division>_<year>_<month>.dat        The HOUSEHOLD symbol is a literal name, <division> is the        division name, <year> is the creation year; in one embodiment,        the year is given as a four digit number (YYYY). The <month>        symbol represents the creation month, two digits zero        padded (MM) and .dat is a literal. An exemplary field        description table for the header portion of the household data        records is illustrated in Appendix AV of previously incorporated        U.S. Provisional Patent Application Ser. No. 61/260,346. An        exemplary field description table for the detail portion of the        household data records as illustrated by Appendix AW thereof as        well.

Household data records are obtained in a join. The join has the headerand the detail in one line. The term “join” refers to an operation wheretwo or more separate data tables are “joined” via a primary key (orcommon data element) and specific data elements from each table arecombined into a new table with the desired results. Logic may beimplemented in embodiment, in Openet programming language and used toseparate the header and the footer. Appendices AX, AY and AZ ofpreviously incorporated U.S. Provisional Patent Application Ser. No.61/260,346 illustrate a sample household file, sources for datacontained in the header and detail portions of the household data files,respectively.

The station data describes all the programs and channels that were shownduring the period. The station data files are named according to thefollowing convention:

-   -   STATION_<division>_<region>_<year>_<month>.dat        In the above naming convention, “STATION” is a literal name,        <division> is the division name, and <region> is the region        name. In one embodiment, the region name corresponds to the        lineup being used to map the channel ID. The <year> symbol is        the creation year, four digits (YYYY); the <month> symbol is the        creation month, two digits (MM); and “.dat” is a literal.        Appendix BA of previously incorporated U.S. Provisional Patent        Application Ser. No. 61/260,346 illustrates an exemplary field        description table for the header portion of the station data        records. Appendix BB thereof illustrates an exemplary field        description table for the detail portion of the station data        records. Appendix BC thereof gives a sample station data file.        Appendices BD and BE thereof illustrate sources for data        contained in the header and detail portions of the station data        files, respectively.

The tuning data report describes all the tuning events that werecollected for a given division during the period. The tuning data filesare in one embodiment named according to the following convention:

-   -   Tune_<division>_<year>_<month>_<day>.dat        In the above naming convention, “Tune” is a literal, <division>        is the division name, <year> is the creation year (four digits,        YYYY), <month> is the creation month (two digits, zero padded,        MM), <day> is the creation day (two digits, zero padded, DD),        and “.dat” is a literal. Appendix BF of previously incorporated        U.S. Provisional Patent Application Ser. No. 61/260,346        illustrates an exemplary field description table for the tuning        data records. Appendix BG thereof gives a sample tuning data        file. Appendix BH thereof illustrates sources for data contained        in the tuning data files.

The above-disclosed files may be delivered to outside entities, such asthe aforementioned National Cable Companies (NCC), via secured FTP orother such protocol. The system 200 has a scheduled task for eachindividual report for each individual division. This advantageouslyallows for asynchronous creation of reports for each division. Theoutput of these reports is controlled with a single control table thatstates the period being collected for that report. An exemplary schemafor the table is illustrated in Appendix B1 of previously incorporatedU.S. Provisional Patent Application Ser. No. 61/260,346.

In one embodiment, daily reports are executed after midnight for recordsprocessed the day before. In another embodiment, for monthly reports,the scheduled task is executed on the first of the month, and processesrecords for the entire month.

Files are delivered using e.g., Secure FTP (SFTP) over a TCP/IP link, orother such mechanism. The system 200 pushes the data files to thedownstream system. The file creation process executes for example once aday in the illustrated embodiment, but file creation may be controlledthrough user-specified parameters. The collected files are stored into adedicated directory on the local file system.

The following exemplary pseudocode describes the process for creatingreports. Although each report executes independently in its ownscheduled task, they comply with the following logic.

Lb_GetControlDates {Report Type, Division} { Get the control dates fromthe control table. If this is for a daily report then If theREPORT_PERIOD for the latest LAST_RUN is less than today's day, Returnthe REPORT_PERIOD value Else Nothing to process Else If theREPORT_PERIOD for the latest LAST_RUN is less than today's month, Returnthe REPORT_PERIOD value Else Nothing to process End If }Lb_UpdateControlDates {Report Type, Division} { If this is a dailyreport, Add 1 to the REPORT_PERIOD field. If the REPORT_PERIOD field isgreater than the end of the month being processed Set REPORT_PERIOD to 1End If Else If this a monthly report, Add 1 to the REPORT_PERIOD field.If LAST_RUN field is greater than 12, then set REPORT_PERIOD to 1 End IfEnd If Insert new entry in the control table with the correlator name,division that was processed, sysdate for the last run, the type ofreport being processed, the parameter for this report and the completedREPORT_PERIOD } Lb_ProcessRecords {Recordset} { For all the records doIf records does not have a header Write record in report format. ElseWrite header with header information off the record set. Write detailoff the detail information off the record set. End If Next }Lb_GetRecords {Control period, Division} { If this is for a daily reportthen Select the record for the specific report for the day plus 1returned  by the control table. Else Select the record for the specificreport for the month plus 1 by the control table. End If Return therecords that were returned by the database } Eh_CreateReport { Controlperiod = Lb_GetControlDates (Report Type, Division) If there issomething to report on then Open a new report file for the division andreport type Recordset = Lb_GetRecords (Control period, Division) Status=Lb_ProcessRecords (Recordset) Lb_UpdateControlDates (Control period,Division) Flush and close report file End If }  © Copyright 2009-2010Time Warner Cable, Inc. All rights reservedVI. Reporting—

One of the salient functions of the SDA/AM 203 is to provide a reportinginfrastructure (based on the data collected from the devices thatprovide usage—namely the SDV tuning events, Video On Demand Events andAd Plays). All the processed usage is stored in a separate entity,called the SDA instance 203, that is used exclusively for the purpose ofreporting. The SDA instance 203 also hosts the reference data needed forcreating ad hoc reports that are used to describe the subscriberbehavior to programs, video or advertisement. All reports are in theexemplary embodiment available for view for 90 days, although otherperiods may be specified.

For the purposes of delivery, the following reports are supported at aminimum: (i) “obtain second-by-second view of household viewership”,(ii) “obtain second-by-second view of household commercial viewership”,and (iii) “obtain view of VOD consumption”. Each of these reports isconstrained by the following attributes where appropriate:

-   -   Subscriber Service Region    -   Media Sale Regions    -   Newtork(s)/Broadcasting Network    -   Day Part    -   Program (irrespective of network)    -   Length of program    -   Advisory Rating    -   AS (e.g., Mystro) Rating    -   Star Rating    -   Genre(s)    -   Zip Codes (or groups)    -   DMA    -   Ad Zones (or groups)    -   Single Anonymous Household    -   Settop Type    -   Settop Capabilities    -   Settop Activities (Namely, recorded programs)    -   Prizm Code    -   CPE Authorization    -   Household Service Mix        -   The Household has video        -   The Household has HSD (high speed data)        -   The Household has Voice (e.g., VoIP)            The above attributes are able to be combined, thereby            providing a means of segmenting the reporting data. All data            provided is also optionally made available through an MSO            secure website, and available for download.            Obtain Second-by-Second View of Household Viewership—

The goal of the “second by second view” of the viewership records is tobe able to query the database with a specific time period, divisionidentifier and service name. The system returns a list of recordscontaining the requisite information. An exemplary embodiment of ahousehold viewership report is illustrated in the table of Appendix BJof previously incorporated U.S. Provisional Patent Application Ser. No.61/260,346. The household viewership report may be built with the fieldsand tables described in the mapping table of Appendix BK thereof aswell. These are collected from the SDA database instance, in that it isdedicated for reporting.

Obtain Second-by-Second View of Household Commercial Viewership—

The goal of the “second by second view” of commercial viewership recordsis to be able to query the database with a specific time period,division identifier and service name in which a givencommercial/advertisement copy was watched. The system returns a list ofrecords containing the information outlined in the exemplary commercialviewership report such as that of Appendix BL of previously incorporatedU.S. Provisional Patent Application Ser. No. 61/260,346. The report isorganized by advertiser, creative and specific campaign. The report ofAppendix BL may, in a further variant, be built with the fields andtables described in the mapping table of Appendix BM of previouslyincorporated U.S. Provisional Patent Application Ser. No. 61/260,346.These are collected from the SDA database instance. The constraintsillustrated in Appendix BN of previously incorporated U.S. ProvisionalPatent Application Ser. No. 61/260,346 may be used to modify theprevious query.

Obtain View of VOD Consumption—

One objective of the “VOD consumption view” is to be able to providedata on how subscribers view VOD assets. An exemplary VOD consumptionreport is shown in Appendix BO of previously incorporated U.S.Provisional Patent Application Ser. No. 61/260,346.

These records are in the exemplary embodiment correlated with theanonymous households, and obey the following constraints:

1. Total time in trick modes

2. Total number of trick modes events

3. Total time in fast forward

4. Number of specific fast forward events

5. Total time in rewind

6. Number of specific rewind events

7. Total time in pause

8. Number of specific pause events

9. Type of VOD (SOD, POD, VOD, Start Over, Quick Clips)

The above report may be built with the fields and tables described inthe exemplary mapping table of Appendix BP of previously incorporatedU.S. Provisional Patent Application Ser. No. 61/260,346. These arecollected from the SDA database instance. The constraints illustrated inAppendix BQ thereof may be used to modify the previous query.

Reporting constrains these reports (via the above-listed constraints).The strategy for applying a constraint includes filtering the usage toone or more given constraint criteria. Exemplary reporting constraintsfor tuning events are given in the table of Appendix BR of previouslyincorporated U.S. Provisional Patent Application Ser. No. 61/260,346.Exemplary reporting constraints for VOD events are given in the table ofAppendix BS thereof. Exemplary advertising event constraints are givenin the table of Appendix BT thereof as well.

VII. Error Recovery—

Records that fail validation or business logic (discussed in greaterdetail below) are in one embodiment stored in error repositories in thesystem 200. Exemplary use and recovery disposition records are describedin the table of Appendix BU of previously incorporated U.S. ProvisionalPatent Application Ser. No. 61/260,346.

The MSO provides facilities to keep the internal reference data current;however, in some instances, records fail because the data isinconsistent with the subscriber information or the network. To accountfor this case, the system 200 provides a recycling mechanism toreprocess records that could not find matching records.

Tuning records may in one embodiment be inserted into a tuning eventtable (e.g., BL_SDV_TUNLOG_TCP_TUN table) from the sources disclosed inthe table of Appendix BV of previously incorporated U.S. ProvisionalPatent Application Ser. No. 61/260,346.

An exemplary method for recycling unguided tuning events according tothe invention is illustrated in FIG. 16. Similarly, an exemplary methodfor recycling VOD events is illustrated in FIG. 17.

VOD records are inserted into the VOD event table (e.g., BL_VOD_EVENTS)from the sources disclosed in the table of Appendix BW of previouslyincorporated U.S. Provisional Patent Application Ser. No. 61/260,346.

An exemplary method of recycling advertising events according to theinvention is illustrated in FIG. 18.

Advertising event records are inserted into an advertising events table(e.g., BL_AD_EVENTS) from the sources listed in the exemplary table ofAppendix BX of previously incorporated U.S. Provisional PatentApplication Ser. No. 61/260,346.

An exemplary method for recycling tuning filtered events is illustratedin FIG. 19.

Tuning filtered events are inserted into a filtered tuning event table(e.g., BL_SDV_TUNLOG_TCP_TUN table from the sources listed in theexemplary table of Appendix BY of previously incorporated U.S.Provisional Patent Application Ser. No. 61/260,346.

Application errors are logged in the existing system 200. These includeprocessing errors, system warning and alerts. Exemplary types of errorsare shown in Appendix BZ of previously incorporated U.S. ProvisionalPatent Application Ser. No. 61/260,346 are in one embodiment capturedand reported.

Error codes such as those listed in Appendix CA of previouslyincorporated U.S. Provisional Patent Application Ser. No. 61/260,346 areadded to the default error codes in the system 200.

VIII. Software Development—

Referring now to FIG. 20, one exemplary implementation of anarchitecture for the interaction between the system 200 and SDA 203 isdescribed. As illustrated, the architecture generally comprises firstserver(s) 2002 and a second server(s) 2004. The first server(s) 2002comprises the various software and hardware components whichcollectively comprise the data collection system 200. The secondserver(s) 2004 comprises at least two functions: (a) a managementfunction 2006 for managing the system 200 running on the first server2002; and an analysis function (or SDA) 203 which analyzes collecteddata. In one embodiment, the second server(s) 2004 comprises an OracleRelational Database Management System (RDMS), the management function2006 and the SDA 203 running as Oracle Instances thereon.

The management function 2006 of the second server(s) 2004 is used tomanage the system 200 including, e.g., advertisement play data from theadvertisement data collector 210 and the VOD data from VOD collector211. The SDA 203 function provides analytics and reporting, as well asproviding files to NCC and other entities (e.g., integration partners).Detail on implementation of these components is provided elsewhereherein.

In one embodiment, the first server 2002 comprises a single serverdedicated to hosting the data collection system 200 processing. In otherwords, according to this embodiment, all of the components of the system200 reside at the operating system level. The system 200 components,such as those listed in the table of Appendix CB of previouslyincorporated U.S. Provisional Patent Application Ser. No. 61/260,346,may be created and deployed in one such implementation.

IX. Statistics—

Collection and output file statistics are collected by a manager entityof the system 200 with the relevant semantics according to the recordsbeing processed or files being delivered. In one embodiment, thestatistics are developed using an Openet FusionWorks Statistics ManagerVersion 2.0, and using the facilities delivered within the collectionsystem 200. By using the framework, business logic compliance withcurrent and future releases of the data collection system 200 isadvantageously assured.

Collection statistics are collected and stored using the aforementionedmanger (such as e.g., the Statistics Manager from SDL 2.0). Eachcollected file from the network elements has statistics such as thoselisted in the table of Appendix CC of previously incorporated U.S.Provisional Patent Application Ser. No. 61/260,346 collected (per file),and are given a definition in SDL. Output file statistics (or file levelstatistics) are also collected and stored for all generated downstreamfiles. The collected output files have, in one embodiment the definitiondiscussed in Appendix CD of previously incorporated U.S. ProvisionalPatent Application Ser. No. 61/260,346.

X. Archiving—

Input files, as collected, are stored into a dedicated directory on thelocal file system. In one embodiment, input files are stored for e.g.,90 days, after which they are deleted. Files that are older than thisspecified age are deleted from the system (e.g., on a daily basis orsome other specified periodicity).

Once a file is parsed successfully, the file is moved to an archivedirectory specified in the collector configuration. An exemplary archivedirectory may be for example of the form:

-   -   /u01/m01/archive/input/<collector name>

Once output files are delivered to the downstream systems, they arestored into a dedicated directory on the local file system. In oneembodiment, output files are stored for 90 days, after which they aredeleted.

Once a file is transferred successfully, the file is moved to an archivedirectory specified in the collector configuration parameter“sp_downstreamArchiveDir”, for example:

-   -   /m01/u02/archive/downstream/<collector name>.

Exception (error, filter & duplicate) files are stored into a dedicateddirectory on the local file system. Exception files are, in oneembodiment, stored for 90 days after which they are deleted,

Once a file is parsed successfully, all downstream and exception filesare completed. If exception files were produced, these files are movedto an archive directory specified in the collector configurationparameter “sp_exceptionArchiveDir”, for example:

-   -   /m01/u02/archive/exception/<collector name>,        XI. Collector—

FIG. 21 illustrates an exemplary interaction between a collector andother components within the system 200. The event flows of FIG. 21 showthe interaction between the collector and correlator and the eventsbeing passed between them.

XII. Business Logic—

The business logic for the subscriber details collector 206, AScollector 210, and/or EPG data collector 214 are implemented, in oneembodiment, utilizing the logic illustrated in FIG. 21. The businesslogic implements rules for the following functions:

-   -   Connect    -   Collect_Statistics    -   Collect_complete    -   Batch_start    -   UDR UDE    -   Parse_Statistics    -   Batch end    -   Disconnect        Exemplary rules for the above business logic are given in        Appendix CE of previously incorporated U.S. Provisional Patent        Application Ser. No. 61/260,346.        XIII. Database Objects—

The system 200, in one embodiment, uses the aforementioned Oracle RDMSfor processing and storing configuration information. The DatabaseObjects and its design are described in greater detail subsequentlyherein.

XIV. Reference Data Loading—

Data that is considered static is loaded directly into the SDA instance203 through e.g., the use of a loader entity or through directinsertion. In one embodiment, the loader entity comprises an Oracle SQLLoader, and/or the direct insertion occurs via SQL*PLUS. Data is thenmanaged by an operator as needed. A plurality of tables (discussedbelow) are created and data loaded from the described source into thecreated tables. If the source is delivered through a spreadsheet, theoperator either converts the spreadsheet to a comma separated value filethat can be conveniently loaded through the loader entity or may beentered by hand as needed.

A SYSCODE to ZIPCODE table (e.g., AM_SYSCODE_ZIPCODE) is used to mapSYSCODES from advertisement plays to zip codes. An exemplary schema forthe SYSCODE to ZIPCODE table is illustrated at Appendix CF of previouslyincorporated U.S. Provisional Patent Application Ser. No. 61/260,346. Inone embodiment, SYSCODE data is delivered in an MSO SDA 203 spreadsheeton the SYSCODE-ZIPCODE work sheet. The schema (Appendix CF) matches thespreadsheet fields' layout, thereby enabling it to be exported to CSV(comma-separated values) and loaded into the SYSCODE to ZIPCODE table.It is noted that the CSV is a simple file format that can beimported/exported between, for example, Microsoft® Excel and datatables.

A DIVISION-to-DMA table (e.g., AM_DIV_DMA) is used to map a DIVISION toa Nielsen Designated Market Area (DMA). The DIVISION to DMA table alsocontains the SYSCODE that belongs to a DMA. An exemplary schema for theDIVISION to DMA table is illustrated at Appendix CG of previouslyincorporated U.S. Provisional Patent Application Ser. No. 61/260,346.The DIVISION data is delivered in the MSO SDA 203 spreadsheet on aDIV_DMA work sheet. The schema (Appendix CG) matches the spreadsheetfields' layout and it can be exported to CSV and loaded into theDIVISION to DMA table. Division names that have a “Contracted” appendedare first processed to have the “−Contracted” tag removed.

A division translation table (e.g., AM_DIV_REG_MAP, or AM_DIV_TRANS)contains division and region codes for all divisions. An exemplaryschema for the translation table is illustrated in Appendices CH and CIof previously incorporated U.S. Provisional Patent Application Ser. No.61/260,346. The translation map is used to translate differentnomenclatures among systems. The data is collected from several sourcesand consolidated into the translation table. In one embodiment, data iscollected from the MAS, Cable Track Spreadsheet, ICOMS division list,and/or the geography data table. The spreadsheet illustrated in AppendixCJ of previously incorporated U.S. Provisional Patent Application Ser.No. 61/260,346 is an exemplary entry in the division translation table.

An equipment table (e.g., AM_EQUIPMENT, or AM_AD_ZONE_SYSCODE) is usedto store information regarding each of a plurality of equipment typesuseful with the present invention. An exemplary schema for the equipmenttable is illustrated in Appendices CK and CL of previously incorporatedU.S. Provisional Patent Application Ser. No. 61/260,346. As shown thetable contains equipment data and peculiarities. The equipment data iscollected in spreadsheets and entered (either manually or automatically)into a worksheet. In one embodiment, the data is entered by hand fromthe STBs—Feature Spec Breakdown.xls.

An advertisement server alignment table (e.g., BL_SVC_ALIGNMENT) is usedto determine the service in which an advertisement event was placed. Anexemplary schema for the advertisement server alignment table isillustrated in Appendix CM of previously incorporated U.S. ProvisionalPatent Application Ser. No. 61/260,346. This advertisement server tableis, in one embodiment, built by hand and managed by an operator. Thedata for the advertisement server alignment table is collected inspreadsheets which are entered (manually or automatically) from the MSOSDA 203 worksheets. The fields of the advertisement server alignmenttable may be populated as illustrated in Appendix CN of previouslyincorporated U.S. Provisional Patent Application Ser. No. 61/260,346.

A geographic alignment table (e.g., AM_GEO_ALIGNMENT, orBL_GEO_ALIGNMENT) is used to place an advertisement event into aspecific geographic location. The geographic alignment table is, in oneembodiment, built by hand and managed by an operator. An exemplaryschema for the geographic alignment table is illustrated in Appendix COof previously incorporated U.S. Provisional Patent Application Ser. No.61/260,346. The data entered into the geographic alignment table iscollected in spreadsheets and entered (by hand or automatically) fromthe MSO SDA 203 worksheets. An exemplary spreadsheet description isillustrated in Appendix CP of previously incorporated U.S. ProvisionalPatent Application Ser. No. 61/260,346.

A time zone table (e.g., AM_TMSTIMEZONE, or TMSTIMEZONE) is used fortime zone determination. The content of the time zone table is loadeddirectly into SDA 203 using a loader entity (such as the aforementionedSQL*LDR). An exemplary schema for the time zone table is illustrated inAppendix CQ of previously incorporated U.S. Provisional PatentApplication Ser. No. 61/260,346. In one embodiment, the time zone tablehas the dimension used by the TMS tables for time zone determination.According to this embodiment, data is directly loaded from thetimezonerec.txt delivered by TMS using SQL*LDR.

A TMS translation table (e.g., AM_TMSTRANSLATION or TMSTRANSLATION) maybe utilized in the instance data is received from a TMS entity. The TMStranslation table translates words that are commonly used in TMS. Thecontent of the TMS translation table is loaded directly into SDA 203using SQL*LDR. An exemplary schema for the TMS translation table isillustrated in Appendix CR of previously incorporated U.S. ProvisionalPatent Application Ser. No. 61/260,346. Data is directly loaded to theTMS translation table, in one embodiment, from the statrec.txt deliveredby TMS using SQL*LDR.

A demographic table may be utilized to match zip codes to generalizeddemographic description of the area. In one embodiment, the table mayutilize Prizm® market segmentation data, e.g., generate an AM_PRIZM_ZIP4table. The content of the demographic table is loaded directly into SDA203 using a loading entity (such as e.g., SQL*LDR). An exemplary schemafor generating a demographic table is given in Appendix CS of previouslyincorporated U.S. Provisional Patent Application Ser. No. 61/260,346.Data may be directly loaded from the ZIP4_PNE delivered by TMS usingSQL*LDR to generate the aforementioned demographic data table.

In another embodiment, the demographic table comprises an AM_PRIZM_ZIPtable, which uses a shortened ZIP code (i.e., without the additionalfour digits). An exemplary schema for generating a demographic table isgiven in Appendix CT of previously incorporated U.S. Provisional PatentApplication Ser. No. 61/260,346.

A Mystro Digital Navigator identification table (e.g., BL_STB_FIRMWARE)may be used to match a given MAC address to its respective firmware inone embodiment. Appendix CU of previously incorporated U.S. ProvisionalPatent Application Ser. No. 61/260,346 illustrates exemplary schema forthe MDN identification table.

There are two files which are used in the exemplary embodiment topopulate the MDN identification table: lch-mdn-macs.txt.gz, andlch-odn-macs.txt.gz. One file includes the MAC addresses that belong toMDN CPE and the other one the addresses for the ODN.

Exemplary file formats managed through the collectors (202, 204, 206,208, 210, 212, and 214) are described in Appendix CV of previouslyincorporated U.S. Provisional Patent Application Ser. No. 61/260,346.File formats for reference data and database tables that are used forcollection are described in the below.

Table 1 of Appendix CV shows exemplary file formats for SDV tunerevents. Tables 2-4 illustrate exemplary file formats for AS servicesrecords, line-up and channel records, respectively. Tables 5-6illustrate exemplary file formats for EPG schedule records andprogramming records, respectively. Tables 7-8 illustrate exemplary fileformats for subscriber billing and details records, respectively.

XV. Audience Measurement Interface—

A specific interface for retrieving Audience Measurement events fromsample data sets prior to the transmission to the SDA 203 is discussedherein. Three types of data are collected via the interface: (i)household data, (ii) channel information, and (iii) tuning data. Ingeneral, data is presented in pipe delimited files to a read-only SFTPsite to which parties with whom the MSO has completed the propercommercial and privacy agreements are given access.

Reference File—MAS Service Information

The exemplary MAS service information reference file is one of two filesused to correlate a station identification (STATION_ID) in the STATIONSUMMARY table to a TMS station identification (TMS STATION ID) forgeneric correlation of anonymous viewing across MSOs. The MAS serviceinformation reference files are delivered on e.g., a daily basis. MASservice information reference files are, in one embodiment, in thefollowing format:

-   -   AM_NCC_MAS_SERVICE_[DIVISION]_[DATE]_[FILE_CREATE_DATETIME].dat        The data within the file is formatted as follows (all fields are        present, but some may be blank): DIVISION_CODE|GUIDE_ID|.        Exemplary MAS service information reference file data        descriptions are illustrated by Appendix CW of previously        incorporated U.S. Provisional Patent Application Ser. No.        61/260,346,        Reference File—TMS Station Information

This TMS station information file is the second of the two files used tocorrelate a station identification (STATION_ID) in the STATION SUMMARYtable to a TMS station identification (TMS STATION ID) for genericcorrelation of anonymous viewing across MSOs. In one embodiment, the TMSstation information file is delivered on a daily basis. TMS stationinformation files comprise the following format:

-   -   AM_NCC_TMS_STATION_[DIVISION]_[DATE]_[FILE_CREATE_DATETIME].dat        The data within the file is formatted as follows (all fields are        present, but some may be blank):    -   STATION_NUM|STATION_TIME_ZONE|STATION_NAME|STATION_CALL_SIGN|.        Appendix CX of previously incorporated U.S. Provisional Patent        Application Ser. No. 61/260,346 illustrates exemplary TMS        station information reference file data descriptions.        Reference File—Universe Data

Universe data is refreshed on a daily basis in one embodiment. Theuniverse data reference file comprises the following format:

-   -   AM_NCC_UNIVERSE_[DIVISION]_[DATE]_[FILE_CREATE_DATETIME].dat        The data within the file is formatted as follows (all fields are        present, but some may be blank): # CABLE HOMES HH|# DIG HH|ZIP|.        Exemplary universe data reference file data descriptions are        illustrated in Appendix CY of previously incorporated U.S.        Provisional Patent Application Ser. No. 61/260,346.        Household Summary Data

Household summary data is in one embodiment, refreshed on a daily basis.Household summary data is in a file with the format:

-   -   AM_NCC_HOUSEHOLD_SUMMARY_[DIVISION]_[DATE]_[FILE_CREATE_DATETIME].dat        Here, the DIVISION symbol is the name of the division (e.g.        Charlotte); DATE represents the date of the file in the format        YYYYMMDD; and FILE_CREATE_DATETIME is the date and time when the        file was created. The data within the household summary data        file is formatted as follows (all fields are present, but some        may be blank):

HH GUID|HH REGION|HH ZIP|HH_RETAILUNITCODE|VIDEO INSTALL DATE|HH HASPHONE|PHONE INSTALL DATE|HH HAS HSD|HSD INSTALL DATE|HH HASMOBILE|MOBILE INSTALL DATE|HH STB COUNT|HH PRIZM CLUSTER|.Each data element may in one embodiment represent the data listed in thetable of Appendix CZ of previously incorporated U.S. Provisional PatentApplication Ser. No. 61/260,346.

The following rules are applied in one embodiment of the invention tohousehold summary data:

-   -   1. A query is executed one time each day to collect this        information    -   2. Tuning records that do not have HH GUIDs matching the data        herein is discarded    -   3. No historical data is provided (history can be tracked in the        destination system based on differences between days).        Household Detail Data

In one embodiment, household detail data is refreshed on a daily basisin a file with the format:

-   -   AM_NCC_HOUSEHOLD_[DIVISION]_[DATE]_[FILE_CREATE_DATETIME].dat        DIVISION is the name of the Division (e.g. Charlotte), DATE is        in the format YYYYMMDD, and FILE_CREATE_DATETIME is the date and        time when the file was created. The data within the file is        formatted as follows (all fields are present, but some may be        blank):

HH_GUID|STB GUID|STB INSTALL DATE|STB IS HD|STB IS DVR|STB HAS PREMIUMVIDEO|SVC TIER DETAIL|TV CAPABLE|ITV OPT IN|.Each data element may in one embodiment represent the data in the tableof Appendix DA of previously incorporated U.S. Provisional PatentApplication Ser. No. 61/260,346.

The following rules are applied to household detail data:

-   -   1. The report is generated one time each day    -   2. Tuning records that do not have STB GUIDs matching the data        herein is discarded    -   3. No historical data is provided (history can be tracked in the        destination system based on differences between days).        Station Summary Data

Station summary data is refreshed on a daily basis in one embodiment.The station summary data is represented as follows:

-   -   AM_NCC_STATION_SUMMARY_[DIVISION]_[DATE]_[FILE_CREATE_DATETIME].dat        DIVISION is the name of the Division (e.g. Charlotte), DATE is        in the format YYYYMMDD, and FILE_CREATE_DATETIME is the date and        time when the file was created. The data within the file is        formatted as follows (all fields are present, but some may be        blank):

STATION ID|STATION LONG NAME|RETAILUNITCODE|CHANNEL NUMBER|NUMBERPROGRAMS|.Each data element may in one embodiment be represented by the table ofAppendix DB of previously incorporated U.S. Provisional PatentApplication Ser. No. 61/260,346.

The following rules relate to the service summary data:

-   -   1. A query is executed one time each day to collect this        information    -   2. Tuning records that do not have STATION IDs matching the data        herein are discarded        Station Detail Data

In one embodiment, station detail data is refreshed on a daily basis ina file with the format:

-   -   AM_NCC_STATION_DETAIL_[DIVISION]_[DATE]_[FILE_CREATE_DATETIME].dat        DIVISION is the name of the Division (e.g. Charlotte), DATE is        in the format YYYYMMDD, and FILE_CREATE_DATETIME is the date and        time when the file was created. The data within the file is        formatted as follows (all fields will be present, but some may        be blank): STATION_ID|PROGRAM NAME|DESCRIPTION|START|STOP|GENRE        1|GENRE 2|GENRE 3|GENRE 4|GENRE 5|IS FIRST RUN|IS A PREMIERE|IS        A SPECIAL|RATING|. Exemplary data elements of the above format        are illustrated in Appendix DC of previously incorporated U.S.        Provisional Patent Application Ser. No. 61/260,346.        The following rules may be applied to the station detail data:    -   1. A query is executed one time each day to collect this        information    -   2. Tuning records that do not have STATION IDs matching the data        herein are discarded        Tuning Data

In one embodiment, tuning data is refreshed on a daily basis in a filewith the format:

-   -   AM_NCC_TUNING_EVENT_[DIVISION]_[DATE]_[FILE_CREATE_DATETIME].dat        DIVISION is the name of the Division (e.g. Charlotte), DATE is        in the format YYYYMMDD, and FILE_CREATE_DATETIME is the date and        time when the file was created. The data within the file is        formatted as follows (all fields will be present, but some may        be blank):

HH GUID|STB GUID|EVENT_TYPE|STATION ID|TUNER ID|IS RECORDING|IS PPV|ISOD|START|STOP|TUNE_OUT_TYPE|.Each data element may in one embodiment represent the data listed inAppendix DD of previously incorporated U.S. Provisional PatentApplication Ser. No. 61/260,346.

The following rules may be utilized for determining what constitutes atuning event:

-   1. A Tuning Event is persisted when the MAIN tuner on a STB is tuned    to the same channel for three consecutive seconds without    interruption.-   2. A Tuning Event is persisted when the PIP tuner on a STB is tuned    to the same channel for three consecutive seconds without    interruption.-   3. A Tuning Event starts when a STB actively requests to view a    channel.-   4. A Tuning Event ends when a STB actively requests a different    channel than the one which is currently streaming.-   5. A Tuning Event ends when a STB enters a Standby state (the STB is    actively turned off)-   6. It is possible for a single STB to register two concurrent Tuning    Events if the box is in both MAIN and PIP mode at the same time.    Advertisement Insertion Data

In one embodiment, advertisement insertion data is refreshed on a dailybasis in a file with the format:

-   -   AM_ADS_[DIVISION]_[DATE]_[FILE_CREATE_DATETIME].dat        DIVISION is the name of the Division (e.g. Charlotte), DATE is        in the format YYYYMMDD, and FILE_CREATE_DATETIME is the date and        time when the file was created. The data within the file is        formatted as follows (all fields will be present, but some may        be blank):

SPOT_START_TIME|SPOT_END_TIME|SPOT_LENGTH|VERIFICATION_STATUS|CAMPAIGN_ID|CAMPAIGN_DESCRIPTION|CUSTOMER_ID|CUSTOMER_NAME|AGENCY_ID|AGENCY_NAME|COPY_ID|SPOT_TYPE_ID|AD_TYPE_ID|NETWORK_ID|RETAILUNITCODE|SYSCODE|STATION_ID|.Each data element may in one embodiment be represented by the data inthe table of Appendix DE of previously incorporated U.S. ProvisionalPatent Application Ser. No. 61/260,346.

The following rules may be utilized in delivering advertisement plays:

-   -   1. Only include in this extract records with a Verification        Status of “aired”        VOD Data        VOD data is refreshed on a daily basis in a file with the        format:    -   AM_VOD_[DIVISION]_[DATE]_[FILE_CREATE_DATETIME].dat        DIVISION is the name of the Division (e.g. Charlotte), DATE is        in the format YYYYMMDD, and FILE_CREATE_DATETIME is the date and        time when the file was created. The data within the file will be        formatted as follows (all fields will be present, but some may        be blank):

HH_GUID|STB_GUID|ASSET_ID|ASSET_TITLE|TIME_AFTER_MIDNIGHT|STREAM_(—)START_TIME|STREAM_END_TIME|STREAM_DURATION|CALENDER_ID|TIME_ID|LAST_ERROR_CODE|HAS_ERROR|IS_MENU_STREAM|LAST_ERROR_DESC|PAUSE_VIEWING_SECS|FF_VIEWING_SECS|RW_VIEWING_SECS|NBR_OF_PAUSES|NBR_OF_RW|NBR_OF_FF|NBR_SESSIONS|NBR_COMPLETE_VIEWS|LAUNCH_CATEGORY|TV_RATING|MOVIE_RATING|POSTED_COST|ACTUAL_COST|The data elements may in one embodiment be represented in the data ofAppendix DF of previously incorporated U.S. Provisional PatentApplication Ser. No. 61/260,346.

The following rules may be utilized to determine what constitutes a VODreport:

-   -   1. A VOD Report is the total number of seconds that a subscriber        took to view a VOD asset either to (1) completion or (2)        abandonment—the lease period expired before the view was        complete.    -   2. A VOD Report may consist of multiple actual VOD sessions        (e.g. if a subscriber stopped the VOD play to watch something        else and returned to it later).    -   3. A subscriber may have viewed a particular VOD more than one        time during a VOD Report (e.g. a subscriber may have watched a        movie two times in its entirety during the lease window for the        particular VOD.        XVI. Network Architecture—

Referring now to FIG. 22, another exemplary network architectureimplementation for the data collection 200 and data analysis (SDA)systems 203 is given.

At the collection system 200, the collection measurements are based onthe following attributes of the usage:

-   -   Subscriber provisioned division    -   Time spent in a particular stream.    -   Service ID that identifies the channel being watched    -   Advertisement spots that were played, detailing the Asset,        Channel, date, start time and duration,    -   Type of viewing (Picture in Picture, recording, live)        The customer/subscriber may perform the following usage        scenarios:    -   Watch a TV channel    -   Watch a TV channel in which local advertisement is inserted.    -   Record a TV show in an integrated DVR    -   Watch a TV stream in a Picture in Picture window from two        different sources    -   Change Channels    -   Turn the CPE on/off    -   Disconnect the CPE    -   CPE goes into a zombie status.    -   Order and watch a video on demand    -   Watch a TV advertisement        In one embodiment, the record measurements are based on the        following attributes of the usage as recorded by the SDA        Instance:    -   For each of above Use Cases, the user is able to constrain the        output view via various elements. A constrained view is simply        an aggregation of views based on a subset of the population that        meets a particular set of constraints (e.g., number of people        viewing between 4:00 PM and 6:00 PM with content that has an        Advisory Rating of—TV-MA and a Star Rating of 4 stars). The list        that follows specifies the various ways that a view must be        constrainable (e.g., for July 2009). Note: Each Use Case may        describe incremental constraint ability.    -   Video Regions    -   Media Sales Regions    -   Network(s)    -   Daypart    -   Program (irrespective of network)    -   Length of program    -   Advisory Rating(s)    -   Mystro Rating    -   Star Rating    -   Genre(s)    -   Zip Code (or groups thereof)    -   DMA (or groups thereof)    -   Ad Zone (or groups thereof)    -   Anonymous Household    -   Settop Type    -   Settop Capability    -   Settop Activities        -   The program was being recorded on a DVR settop        -   The program was being watched on multiple CPE in the            household    -   EPG Version    -   Prizm Code    -   CPE Authorization    -   Household Service Mix        -   Video Packages (e.g. has HBO, does not have Cinemax)        -   HSD Packages        -   Voice Packages    -   Any Boolean combination of the above constraints        Furthermore, the agency may perform the following usage        scenarios:    -   Purchase an advertising window    -   Purchase targeted advertising spots for a specific demarcation        zones.

FIG. 23 gives yet another exemplary architecture for the collection andanalysis of data. The architecture generally comprises a server used tocollect and modify the collected usage from the SDV, EPG and theadvertisement data sources. The collected usage data is normalized,enriched and anonymized. There are two databases that are hosted at thesecond server (i) a mediation system and (ii) an SDA 203 system.

The SDV 203 server is used to collect tuning and streaming events. Inone embodiment, the SDA 203 system comprises a Cisco SDV Server. Thesecollected tuning and streaming events are used to, inter alia, identifyuser interactions with the network.

Advertisement data is collected from the network. In one embodiment, theadvertisement data collector comprises an OpenTV Eclipse LoggingPlatform. The advertisement data collector is used to collect data aboutadvertising spots that were successfully delivered, the time they werescheduled and the time they were aired. The collected advertisement datais correlated with the SDV server records and used to determine thenumber of users that watched in the reporting layer (SDA 203). Theadvertisement data collector delivers advertisement data through thenetwork.

VOD records are also collected and delivered through the network. In oneembodiment, the VOD records are collected from an Everstream collector.VOD records are correlated with the provisioning data in order toprovide enriched records to the reporting layer (SDA 203).

The provisioning data is created from the billing systems for thosedivisions that are be handled by the system 200. In one embodiment,creation of the provisioning data occurs according to process similar tothose used in digital phone or high speed data mediation.

The application server collects information including e.g., a masterchannel list and a description of the channel lineup.

The advertisement data source inserts advertisement into a televisionchannel or program stream. Advertisement data records are collected bythe advertisement data collector.

Appendix DG of previously incorporated U.S. Provisional PatentApplication Ser. No. 61/260,346 illustrates exemplary collectioninterface which may be utilized with the present invention. Appendix DHthereof illustrates exemplary downstreams for use with the presentinvention.

XVII. SDV Collectors—

SDV collectors 204 are used for the collection of SDV data. In oneembodiment, the SDV collector 204 comprises a Cisco SDV Collector. Forexample, there may be 8 instances of this server per cluster, 1 divisionto collect in total—7 are active, and 1 standby. Only tuner events arecollected through the SDV collectors.

The instances naming convention is as follows:

-   -   <div>_SDV_<type>_<method>_<##>        In the above naming convention, they <div> symbol represents the        division code, the _SDV_ is taken literally. The <type> symbol        represents the type of collector (for example, TUN). The        <method> represents the method of collection (for example, TCP)        and the <##> symbol is a two digit identifier. A sample instance        name is “B23_SDV_TUN_TCP_01”.        Protocol—

The direct connection protocol is through a TCP/IP link. The system 200acts as a TCP client that attaches to a SDV server port. In oneembodiment, there are 5 server ports open per usage type, 15 in totalper SDV. There is also a UDP multicast port available as well. Serversdeliver records in Log 2 Format. Streamed content is collected in realtime.

Collection—

Stream collection via the SDV collector 204 occurs according to thefollowing steps:

-   -   Attach to the SDV server tuner event TCP port.    -   Read records that are being broadcasted at the TCP port.    -   Store stream in buffer if needed    -   Parse and process records from the stream.        Exception Handling—

The SDV TCP collector 204 maintains awareness of its connection statuswith the SDV server data source 207. If the connection between the SDVdata source 207 and the SDV collector 204 is severed, the collector 204records when the connection was severed and tries to reconnect with theSDV server 207. The collector 204 attempts a configurable number oftimes, defaulted to “infinity” in a configurable time interval. If thecollector 204 reconnects, the collector 204 records the time in which itreconnected to the SDV data source 207 and continues processing.

Since the connection is unidirectional and the SDV server 207 does nothave an internal buffer, it is assumed that the records that were notcollected during the outage are lost. In one embodiment, the lostrecords are rebuilt later on using SDV logs.

Input Data Format and Archiving—

Collected files are encoded in plain text, comma separated value in Log2 format. The record format is described in detail below. Streams arenot archived.

Business Logic—

The collected tuner events describe its state according to its “Event”and “Tuner Use” fields. These fields are used to map the viewingimpression state as described in the state diagram of FIG. 24.

The tables of Appendix DI of previously incorporated U.S. ProvisionalPatent Application Ser. No. 61/260,346 illustrate exemplary tuningevents interpretations.

Detecting Tuner Event States—

A method of detecting tuner event states according to the Log 2specifications is now described.

Decoding

The system decodes the records based on the schema described hereinbelow. Data from the tuner event feed is collected as received. Tunerevents are decoded by interpreting the defined schema. SDV Sessions arecreated for each MAC Address, tuner identification number (tuner id) andtuner use combinations. Each of them is interpreted as an individualsession. These SDV sessions provides the building blocks to create aviewing impression. The system supports tuning events from CPE havingthe ODN and the MDN firmware. The event field from the tuner event isused to determine the length and disposition of the tuning event.

One method for detecting an SDV session is discussed herein. TCP recordsare collected as they are streamed by the SDV server. For all collectedtuner events. If the record is a SDV Event (Event Type=1) or Non SDVEvent (Event Type=2) it is interpreted as the start of a SDV session fora specific CPE MAC Address, tuner number and tuner use. Continuetraversing through the list until a terminating tuner event is found forthe CPE MAC Address, tuner identification (tuner id) and tuner useinitiated the tuner event. A tuner event that has one of these eventtypes is considered to be a terminating event: Inactive (6), Program NotFound (7), Aged (3), Scheduled (4), Slated (5), Any Event Type but TunerUse=Failure(128). If a terminating event has been found, the followinginformation is stored for the purposes of completing the SDV session:

-   -   Previous Channel    -   Start Timestamp    -   End Timestamp    -   Duration between events.    -   Type of Event    -   Tuner Use        Next, the SDV session is enriched with the provisioning data by        e.g., using the MAC Address as a key. After the SDV session has        been closed, continue creating SDV sessions for all incoming        records as they are sent by the SDV server 207. The collection        system 200 parses the tuner use bit flags according the Log 2        specifications. SDV sessions that are open but do not have a        matching close impede the creation of a viewing impression, and        thus are considered “zombies”.

In one embodiment, the time that a given SDV session expires isconfigurable, and defaulted for 24 hours.

If an SDV session never finishes and a new SDV session starts for thesame MAC Address, tuner identity (tuner id) and tuner use, the previousSDV session is closed with a potential zombie status. This situation canoccur when a connection to the SDV server 207 is dropped and restored.SDV sessions are converted into viewing impressions that match the statediagram described in FIG. 24. These viewing impressions contain a fieldnamed “State Id” that is populated according to, for example, theconditions listed in the table of Appendix DJ of previously incorporatedU.S. Provisional Patent Application Ser. No. 61/260,346.

Validation Logic

The system 200 validates that the usage follows the format described inelsewhere herein. In one embodiment, the same fields used for duplicatedetection are validated.

Duplication Check Logic

The system 200 provides the ability to verify that SDV records are not aduplicate of any other raw usage records. The following records are usedin one embodiment to detect duplicates: Date, Time, Event, UID/MAC,Source Id, and LUA.

Enrichment Logic

The system 200 enriches the provisioning data with the tuner events toobtain the household information. The system 200 anonymizes tunerrecords by converting the MAC address into a GUID according to the oneor more anonymizing algorithms (discussed herein below).

The system 200 also enriches with the network data by selecting datafrom the master channel table. The result of the query is a unifyingservice identification number that is used to correlate the networkinformation. The following steps are used in one embodiment to correlatea tuning event to the subscriber data:

-   -   Subscriber information is collected and stored for the purpose        of determining the user demographics for a tuning event.    -   The system 200 uses the MAC address from the correlated viewing        event, which was originally obtained from the tuner event, to        search for a matching record in the collected provisioning feed.    -   If a match is found, the MAC address is anonymized and the        enriched subscriber data is added to the viewing event. The        following data items are added to the viewing event: Division,        CPE ID, Zip Code, Anonymized MAC, and Anonymized User ID.    -   If a match is not found, the viewing impression is stored in an        unguided table and reprocessed. When the record is reprocessed,        the MAC address is anonymized and stored as a valid viewing        impression. In one embodiment, Openet provides a configurable        aging mechanism that is defaulted for 7 days.    -   Processed records are delivered to SDA according to the        specifications described below.        The program name is obtained by correlating the source ID of the        viewing event with the source ID of the programming event. The        resultant correlation is the Service ID.        Filtering Logic

Records that the computed tuning duration is less than a configurablenumber of seconds, defaulted to one second are filtered. These recordsare processed and correlated with the programming and provisioningtables. At the end these records are anonymized. Records that werefiltered because of a short duration have the ability to be reprocessedif needed. The duration criteria that is used for reprocessing isconfigurable and defaulted to one second.

Mapping Logic

Tuner, Programming and Stream events are staged in one embodiment.Viewing Impressions are mapped into a new table structure in the SDA-AMinstance 203 that has the information used to describe for how long asubscriber has watched a program. These fields include:

-   -   Anonymized Customer Mac Address (CPE_GUID)    -   Anonymized Subscriber Account Number (CUSTOMER_GUID)    -   Tuner Number (From Mac Address, Tuner Number)    -   Start_Date (“Current Record Date/Time”)    -   End_Date (“Last Record after all related tuning events have been        added”)    -   Total View Time (Duration)    -   Channel_ID (From Source ID correlating with Programming ID)    -   Division_ID (From correlation with subscriber data)    -   Previous Channel ID (Stored during process)    -   Is Recorded (From Tuner Use)    -   Is PIP (From Tuner Use)    -   Is PPV (From Tuner Use)    -   Source Id (From Source Id)    -   State Id (Computed)        Downstream

Downstreams will be discussed in greater detail subsequently herein.

XVIII. SDV Log Collectors—

The system 200 normally collects and processes SDV data from the TCPconnection (discussed above). However, in case that the collector 204disconnects, all records through that period will be lost. As a way ofrecovering these records, the collection system 200 also collects andprocesses SDV logs. The records collected from the SDV logs are used torebuild lost TCP records. After the desired period has been recovered,the SDV log records can be disposed.

In one embodiment, The SDV collector 204 for log files comprises a CiscoSDV collector. In another embodiment, there are 8 instances of thisserver per division, 2 divisions to collect in total, 7 are active and 1stand by.

The instances naming convention is as follows:

-   -   <div>_SDV_<type>_<method>_<##>        According to the above naming convention, the <div> symbol        represents the division code, the _SDV_(—) is a literal, the        <type> symbol represents the type of collector (such as e.g.,        TUN), the <method> represents the method of collection (such as        LOG) and the <##> is a two-digit identifier. A sample instance        name is “B23_SDV_PRO_LOG_01”.        Collection Protocol—

In one embodiment, the log based collection protocol is FTP over TCP/IPlink. Collection and delivery of the SDV Log files is handled by thecollection system 200. The collection system 200 pulls those files fromthe SDV, and handles them internally by its collectors. Files are notcompressed in the exemplary implementation.

Collection Frequency—

Log content is collected once every 24 hours. This is because log filesat the SDV server close at the end of the day. The system 200 waitsuntil the log files are closed before collecting them.

Collection File Naming Conventions—

The file naming convention on the remote host is as follows:

-   -   sdv_###.###.###.###_<type>YYYYMMDD        According to the given naming convention, the sdv_(—) symbol is        a literal, ### symbols represent a number from 1-255, unpadded,        the <type> symbol is the stream type (programs|streams|tuners)        and the YYYYMMDD symbol represents the date by year, month and        day. The following are examples of data filenames:

sdv_10.120.23.100_programs20091212

sdv_10.157.78.233 streams20081106

sdv_10.157.78.233_tuners20081109

Validation: Filename Duplication Check—

The period of time for duplicate check is a configurable parameter.Initially, in one embodiment, the system 200 may be configured to checkduplicate files in the range of 1 to 5 days.

Collection Steps for File Collectors—

The following steps are performed by the system 200 once the files havebeen collected from the SDV data source 207:

-   -   Retrieve files from the SDV Server after they have been closed        at midnight.    -   Files are moved to local AC directory    -   File duplicate check    -   Parse and process the files.    -   Raw records are stored.    -   Once a night a scheduled task determines if a TCP disconnect        occurred during the day. This scheduled task reads the SDV        server log records and re-creates the viewing events for the        day.        Input Data Format—

File is encoded in plain text, comma separated value. It applies to bothtypes of collectors. The record format is described in detail below.

Archiving—

In one embodiment, the system 200 maintains files in archive for apredetermined period (e.g., 7 days).

Business Logic—

The collected tuner events describe its state according to its Event andTuner Use fields. These fields are used to map the viewing impressionstate as described in the state diagram of FIG. 25.

The tuning events are interpreted, in one embodiment, as illustrated inthe tables of Appendix DK of previously incorporated U.S. ProvisionalPatent Application Ser. No. 61/260,346.

Detecting Tuner Event States—

A method of detecting tuner event states according to the Log 2specifications is now described.

Decoding

The system 200 decodes the records based on the schema described below.Data from the tuner events is stored as read and not processed until ascheduled task executes to recreate any missing viewing events. Theseare sorted by user ID, tuner date and time. Tuner events are decodedaccording to the defined schema. Viewing impressions are created onlyfor the records that were lost for the time that the collector was notreceiving records from the SDV server 207. The connection lost may bemore than once during the day. The system 200 recreates those recordsfor each period.

The event field from the tuner event is used to determine the length anddisposition of the viewing impression. These records are sorted inascending order by date and time.

SDV sessions are created for each MAC address, tuner ID, and tuner usecombinations. Each of them is interpreted as an individual session.These SDV sessions provide the building blocks to create a viewingimpression.

The system supports tuning events from CPE having the ODN and the MDNfirmware.

The method for determining viewing impressions in order to rebuildmissing records is now discussed. First, the collected tuner events aresorted by user ID, Date and Time for the period that needs to berebuilt. For all sorted tuner events. If the record is a SDV Event(Event Type=1) or Non SDV Event (Event Type=2) interpret it as the startof a viewing impression for a specific CPE GUID, Tuner Number and TunerUse. Continue traversing through the list until a terminating tunerevent is found for the CPE GUID, Tuner Id and Tuner Use that initiatedthe tuner event. A tuner event that has one of these event types isconsidered to be a terminating event: Inactive (6), Program Not Found(7), Aged (3), Scheduled (4), Slated (5), and Any Event Type but TunerUse=Failure (128).

If a terminating event has been found the following information iscomputed for the SDV Session. This includes:

-   -   Previous Channel    -   Start Timestamp    -   End Timestamp    -   Duration between events.    -   Type of Event    -   Tuner Use

After the SDV Session has been closed, continue traversing the listuntil a new SDV Event (Event Type=1) or Non SDV Event (Event Type=2) isfound. This is repeated for all tuning events.

The system 200 parses the tuner use bit flags according the Log 2specifications.

SDV Sessions that open but do not have a matching close for the periodof the outage are terminated as inactive (6). This takes care of anyinactive records that were disregarded during the TCP collection. Theresultant records are the “raw” viewing impressions. All SDV sessionsare stored and used for recovery purposes as explained below.

SDV sessions are converted into viewing impressions that match the statediagram of FIG. 25. These viewing impressions contain a State ID that ispopulated according to e.g., the conditions of Appendix DL of previouslyincorporated U.S. Provisional Patent Application Ser. No. 61/260,346.Validation Logic

The system 200 stages the collected records. In one embodiment, the samefields used for duplicate detection are validated.

Duplication Check Logic

The system 200 provides the ability to verify that SDV records are not aduplicate of any other raw usage records. The following records are usedto detect duplicates: Tuning Events, Date, Time, Event, UID/MAC, SourceId, LUA, Stream, Program Events, Date, Time, Event, Source, and Name.

Enrichment Logic

The tuning event is correlated with the master channel table to obtainthe service. The network name and the service identity (Service ID) areappended to the viewing event.

While rebuilding viewing impressions, the user ID that was collectedfrom the log is substituted by the one built by the TCP stream. Thesystem 200 matches the viewing impressions that have been collected inboth sources and maps the UID in the log with the GUID and provisioningdata that was computed for the TCP feed and stored internally in a map.This map is used to enrich the GUID and provisioning data to thoseviewing impressions that are being rebuilt to fill in those that weremissing when the connection was lost.

Filtering Logic

Tuning events are filtered; the filtering logic is discussed elsewhereherein.

Mapping Logic

Tuner, Programming and Stream events are staged. The correlated viewingevents have a new table structure that has the relevant Tuner, Streamand Programming fields that are used to describe for how long asubscriber has watched a program. These fields include in the exemplaryembodiment:

-   -   Tuner Number (Tuner State)    -   Start_Date (“Current Record Date/Time”)    -   End_Date (“Last Record after all related tuning events have been        added”)    -   Total View Time (Duration)    -   Channel_ID (From Source ID correlating with Programming ID)    -   Division_ID (From correlation with subscriber data)    -   Previous Channel ID (Stored during process)    -   Is Recorded (From Tuner Use)    -   Is PIP (From Tuner Use)    -   Is PPV (From Tuner Use)        Subscriber related information is not collected because        correlation is not possible until these records have been        rebuilt. Rebuilt records are stored with the TCP collection        tables.        Downstream

Records being collected through log are used for recreating records thatwere lost because the TCP stream was disconnected. See discussion below.

XIX. VOD Collectors—

VOD data is collected via one or more VOD collectors 208. In oneembodiment, the VOD collector comprises an Everstream VOD collector. Inanother embodiment, there is 1 instance of the VOD collector 208.

The instances naming convention is as follows:

-   -   <div>_VOD_<type>_<##>        According to this naming convention, the <div> symbol represents        the division code, the _VOD_ is a literal, the <type> symbol        gives the type of file (e.g., ASSET|EVENT) and the <##>        represents a two-digit identifier. A sample instance name is        “B23_VOD_EVENT_01”.        Collection—

The VOD data source (e.g., VOD server) 211 delivers usage through adatabase connection. The VOD server 211 inserts usage and data ofinterest in a database schema. The collection system 200 reads the usageand data of interest as delivered. VOD data may be delivered once a day.

The following steps are performed by the collection system 200 as partof VOD stream collection:

-   -   Usage is delivered from the VOD server into a usage table in a        specific schema.    -   The data collection system 200 reads the data from the schema        and manipulates the data according to these requirements.    -   The manipulated data is stored for delivery to the SDA instance.        Input Data Format—

Collected data is stored at a storage entity (e.g., an Oracle table).

Protocol—

Collected data is delivered, in one embodiment, through SQL*NET.

Archiving—

The VOD data collector 208 follows archiving procedures and manages theschema used to deliver data to the collection system 200.

Business Logic—

The collected VOD event describes its current state by its related trickmode fields. Exemplary trick modes include play, stop, fast forward,rewind and pause. These fields are used to map the VOD state asdescribed in the state diagram of FIG. 26.

VOD Event Detection—

Exemplary VOD events are described in the table of Appendix DM ofpreviously incorporated U.S. Provisional Patent Application Ser. No.61/260,346.

A method discussing how the system 200 detects these states according tothe values in their respective files is discussed herein.

Decoding

The system 200 decodes the records based VOD schema described hereinbelow.

Validation Logic

The system 200 validates that the usage is based on the schema describedbelow. The following tables are validated according to its data typedefinition:

-   -   LOCATION    -   ORGANIZATION    -   EQUIPMENT    -   VOD_LEASE    -   VOD_STREAM    -   VOD_ASSET    -   VOD_ASSET_TITLE    -   VOD_ASSET_TITLE_GENRE    -   VOD_ASSET_COMP    -   CALENDAR    -   TIME_OF_DAY        Duplication Check Logic

The collection system 200 does not perform duplicate check logic. It isassumed that primary indexes have been already created for each of thetables that are delivered by the VOD collector 208.

Enrichment Logic

The collection system 200 reads the records from the collector 208schema that is needed to create records into the SDA 203 VOD table. Thesystem 200 is required to obtain records from the subscriber data tocomplete it.

The subscriber data is retrieved from the subscriber table by using theMAC address. It returns the subscriber account number that is used tocreate the delivery record. The MAC address and subscriber accountnumber are anonymized by using the anonymization process describedbelow. Records are mapped to match the SDA 203 VOD table. Processedrecords are delivered to SDA 203 according to the specificationsdescribed below.

Mapping Logic

VOD files are staged according to the layout as described below.Correlated VOD events have a new table structure that has the relevantVOD and subscriber fields. These are used to describe the viewing habitsof a given VOD subscriber. The following fields are required to bemapped to the SDA VOD table:

-   -   Customer Guid (From Enrichment)    -   CPE Guid (From VOD Stream Table)    -   Division Id (From Enrichment)    -   Asset Id (From VOD Assets Table)    -   Asset Component Id (From Table)    -   Asset Component Num (From. Table)    -   Time After Midnight (From VOD Stream Table)    -   Stream Duration (From VOD Stream Table)    -   Lease Id (From VOD Lease Id)    -   Stream Start Date (From Calendar)    -   Stream Start Time (From VOD Stream Table)    -   Stream End Time (From VOD Stream Table)    -   Calendar Id (From VOD Stream Table)    -   Time Id From (From VOD Stream Table)    -   Last Error Code (From VOD Stream Table)    -   Has Error (From VOD Stream Table)    -   Is Menu Stream (From VOD Stream Table)    -   Last Error Description (From VOD Stream Table)    -   Pause Viewing Seconds (From VOD Stream Table)    -   Rewind Viewing Seconds (From VOD Stream Table)    -   Fast Forward Viewing Seconds (From VOD Stream Table)    -   Pause Count (From VOD Stream Table)    -   Rewind Count (From VOD Stream Table)    -   Fast Forward Count (From VOD Stream Table)    -   Service Group (From VOD Stream Table)    -   Launch Category (From VOD Stream Table)    -   Is Preview (From VOD Asset Title)        Downstream

Downstreams are disclosed in greater detail subsequently herein.

XX. Advertisement Data—

Advertisement data is collected from an advertisement data source 215 atan advertisement data collector 212. In one embodiment, theadvertisement data collector 212 comprises an Eclipse server. In anotherembodiment, there is one instance of the advertisement data collector212.

Collector Instance Naming Conventions—

The instances naming convention is as follows:

-   -   <div>_ADS_<##>        According to the given naming convention, the <div> symbol        represents the division code, _ADS_(—) is a literal, and the        <##> symbol is a two-digit identifier. A sample instance name is        “B23ADS_01”.        Collection—

The advertisement data server 212 delivers usage through a databaseconnection. The advertisement server 212 inserts usage and data ofinterest in a database schema. The collection system 200 reads the usageand data of interest as delivered. The advertisement data may becollected once a day.

Collection Steps for File Collectors—

The following steps are performed by the collection system 200 once thefiles have been collected from the advertisement collector 212:

-   -   Usage is delivered from the advertisement server into a usage        table in a specific schema.    -   The collection server reads the data from the schema and        manipulates the data according to these requirements.    -   The manipulated data is stored for delivery to the SDA instance.        Input Data Format—

Advertisement data is stored in a storage entity. In one embodiment, thedata is stored in an Oracle table.

Protocol—

In one embodiment, data is delivered through SQL*NET.

Archiving—

The advertisement collector follows its archiving procedures and managesthe schema used to deliver data to the collection system 200.

Business Logic—

Decoding

The collection system 200 decodes the records based on the schemadescribed below.

Validation Logic

The collection system 200 validates the collected records according tothe schema described below. The following tables are validated accordingto its data type definition:

-   -   INVOICE_DETAIL    -   ADVERTISER    -   CONTRACT    -   COPY    -   COPY_GROUP    -   REGION    -   HEADNET    -   RETAIL_UNIT    -   NETWORK    -   PROGRAMS    -   SPOT        Duplication Check Logic

The collection system 200 does not perform duplicate check logic. It isassumed that primary indexes have been already created for each tablethat is delivered by the advertisement data server 212.

Enrichment Logic

The collection system 200 reads the records from that are needed tocreate records into the SDA 203 advertisement server usage tables.Records are mapped to match the SDA 203 advertisement server usagetable.

The advertisement records use the Network Id, Network Instance Id andthe Head End to determine the channel in which a given advertisement wasbroadcasted from a mapping table.

Processed records are delivered to SDA 203 according to thespecifications described in below.

Mapping Logic

The system 200 stages the collected records. In one embodiment, therecords are staged according to the advertisement insertion eventdelivery format and advertisement insertion events reference data fieldsas described in below.

Downstream

Downstreams are disclosed herein below.

XX. EPG Data—

EPG data is collected from an EPG data source 217 at an EPG datacollector 214. In one embodiment, the EPG data source 217 comprises oneor more Tribune Media Systems servers. In another embodiment, there isone instance of delivery from the data source 217.

The instances naming convention is as follows:

-   -   <div>_TMB_<type>        According to the above-disclosed naming convention, the <div>        symbol represents the division code, _TMB_(—) is a literal, and        the <type> symbol represents the file type being managed by the        collector (such as, PROGRAM|SCHEDULE|STATION). A sample instance        name is “B23_TMS_SCHEDULE”.        Collection—

There is one type of collection for these devices; the collection of EPGdata is file based. In one embodiment, the files-based collectionprotocol is FTP over TCP/IP link. EPG data files are compressed usinggzip. The collection system 200 reads “National” files that correspondto the EPG data type.

EPG program and schedule data is collected once every 24 hours in oneembodiment. If no files are available, the data is not collected.Collected data has e.g., 15 days worth of programming schedule.

The file naming convention on the remote host is as follows:

-   -   <tabletype>.txt.gz        In the above naming convention, the <table type> describes the        table (for example, progrec, skedrec, statrec, progrec, etc.),        the .txt.gz symbol is a literal which identifies a data file.

Nightly files contain data extract from the following tables ofinterest:

-   -   Schedule    -   Programming        The following is an exemplary data filename: progrec.txt.gz.

EPG data files are delivered as an MD5 summary file in one embodiment.The termination file indicates when a file has been written and closed.It indicates when all files are ready to be collected. The file namingconvention on the remote host is as follows:

-   -   <tabletype>.txt.gz.sum        The <table type> symbol describes the table (for example,        progrec, skedrec, statrec, progrec, etc.), the .txt.gz.sum        symbol is a literal which identifies a data file summary with        MD5 hash.

Nightly files contain data extract from the following tables ofinterest:

-   -   Schedule    -   Programming        The following is an exemplary data filename: progrec.txt.gz.        Filename Duplication Check—

The collection system 200 does not perform a duplicate check for the EPGdata files because these files have fixed names. Old files are tagged aspart of collection and archived.

Collection Steps for File Collectors—

The following steps are performed by the collection system 200 once thefiles have been collected from the EPG data source 217. First, the EPGdata files are obtained as soon as they are available. Files are thenmoved to a local AC directory, parsed and processed.

Input Data Format

Files are encoded in plain text pipe separated value. The record formatis described in detail herein below.

Archiving—

The collection system 200 maintains the data available online for 7 daysin one embodiment. The data is then stored into a dedicated directory onthe local file system.

Business Logic—

Decoding

The collection system 200 decodes the records based on the schemadescribed below.

Validation Logic

The collection system 200 validates the collected records according tothe schema described in below. The following fields are used for programvalidation: Tf_database_key, Tf_title, Tf_advisory_desc (all of them),and Tf_star_rating. The following fields are used for schedulevalidation: Tf_station_num, Tf_database_key, Tf_air-date, Tf_air_time,Tf_duration, Tf_tv_rating, Tf_sex_rating, Tf_language_rating,Tf_dialog_rating, and Tf_fv_rating. The following fields are used forstation validation: Tf_station_number, Tf_station_timezone,Tf_station_name, and Tf_station_call_sign.

Duplication Check Logic

The collection system 200 provides the ability to verify that the EPGrecords are not a duplicate of any other raw usage records. Thefollowing fields are used for program duplicate detection:Tf_database_key and Tf_title. The following fields are used for scheduleduplicate detection: Tf_station_num, Tf_database_key, Tf_air_date,Tf_airtime, and Tf_duration. The following fields are used for stationduplicate detection: Tf_station_number, Tf_station_timezone,Tf_station_name, and Tf_station_call_sign.

Enrichment Logic

There is no enrichment to be performed for the EPG data feeds. Thesource identifier (Source Id) from the viewing impression is used tointerrogate the MAS database. The result from the query has the guideservice identification which maps to the station number. The stationnumber is used as a means to obtain specific information on programmingand scheduling. Program and Station records are collected but notenriched. These records are used to create reports.

Mapping Logic

The collection system 200 stages the collected records according to theschema described below.

Downstream

Downstreams are described in greater detail below.

Records collected from the EPG data files are stored in the SDA instance203 as collected. Schedule records that are enriched are stored in theSDA instance 203 as collected and enriched.

XXI. Application Server Data—

Application server data is collected from the application server (AS)data source 213 by at least one AS database collector 210. In oneembodiment, the AS 213 is a Mystro™ server device of the type utilizedby the Assignee hereof (discussed in previously referenced co-owned,co-pending U.S. patent application Ser. No. 11/263,015 filed Oct. 2,2002 and entitled “Network based digital information and entertainmentstorage and delivery system”, now published as U.S. Patent ApplicationPublication No. 2003/0208767, which claims priority under 35 U.S.C.119(e) the benefit of U.S. Provisional Application No. 60/377,963 filedon May 3, 2002), although it will be appreciated that otherconfigurations may be utilized with equal success.

In another embodiment, there are 16 instance of the data collector 210.The instances naming convention is as follows:

-   -   <div>_MAS_<type>_<##>        According to the above-disclosed naming convention, <div>        represents the division code, _MAS_(—) is a literal, <type> is        the file type being managed by the collector (such as e.g.,        SERVICE|PROGRAM|CHANNEL), and <##> is a two digit identifier. A        sample instance name is “B23_MAS_SERVICE_01”.        Collection—

The files based collection protocol is FTP over TCP/IP link. Collectionand delivery of the AS data files is handled by the AS data collector210 of the collection system 200. In one embodiment, there is one dataextract from the AS data collector 210 every day.

The file naming convention on the remote host is as follows:

-   -   <fileid>.txt.gz        In the above naming convention, the <fileid> symbol is a file        identifier which may identify “channels” “services” or “lineups”        in lower case, the .txt.gz symbol is a literal and identifies a        data file in text compressed using gzip. Nightly files contain        data extract from the following tables:    -   Services    -   Lineup    -   Channel        The following are examples of data filenames: services.txt.gz,        and channel.txt.gz        Validation—        Filename Duplication Check

File duplication check cannot be performed because files do not have anydistinguishing features. Collected files are moved, processed andarchived. During the archive process, those files are renamed to includea timestamp.

Collection Steps for File Collectors

The following steps are performed by the collection system 200 once thefiles have been collected. First, the AS files are obtained after theyhave been closed at midnight. Files are then moved to local ACdirectory. A file duplicate check, parsing and processing are thenperformed. The files are archived.

Input Data Format

Files are encoded in plain text pipe separated value. The record formatis described in detail elsewhere herein.

Protocol

AS data file collectors 210 use FTP over TCP to collect the files.

Archiving

In one embodiment, the collection system 200 maintains files for 7 days.The collected AS data files are stored into a dedicated directory on thelocal file system.

Business Logic—

Decoding

The system 200 decodes the records based on the schema described below.

Validation Logic

The collection system 200 validates the collected records according tothe schema described herein.

Duplication Check Logic

The system 200 provides the ability to verify that AS data records arenot a duplicate of any other raw usage records.

Enrichment Logic

Records collected from AS data source 213 are not enriched. Theserecords are stored as collected and used for enrichment.

Filtering Logic

As records are filtered; the filtering logic is discussed elsewhereherein.

Mapping Logic

The system 200 stages the collected records according to an applicationserver schema described below.

Records are mapped to the service table as follows:

-   -   Service Id (From Service Feed, Service Id)    -   Guide_Service_Id (From Service Feed, Guide Id)    -   Primary Source Id (From Service Feed, SD Source Id)    -   Secondary Source Id (From Service Feed, HD Source Id)    -   Call Letters (From Service Feed, Service Name)    -   Network Name (From Service Feed, Service Long Name)    -   Service Type (From Service Feed, Service Type)    -   Entitlement Id (From Lineup Feed, Entitlement id)    -   Switched Indicator (From Service Feed, SDV)    -   High Definition Indicator (From Service Feed, HS_SD)    -   Region Id (From Service Feed, Region Name)    -   Division Id (From Services Feed, Division Id)    -   Launch On Boot (To be finalized on the Technical Specifications)    -   Pre Load Catalogs Indicator (To be finalized on the Technical        Specifications)    -   Network Express Indicator (To be finalized on the Technical        Specifications)    -   Channel Transient Indicator (To be finalized on the Technical        Specifications)    -   Immediate Audio Tune Indicator (To be finalized on the Technical        Specifications)    -   Family Tier Indicator (To be finalized on the Technical        Specifications)    -   Start Over Indicator (To be finalized on the Technical        Specifications)

Records are mapped to the channel table as follows

-   -   Channel Id (From Channel Feed, Display Channel)    -   Region Id (From Channel Feed, Region Name)    -   Division Id (From Channel Feed, Division Name)    -   Division Code (From Channel Feed, Division Code)    -   Service Id (From Channel Feed, Service Id)

Records from the Lineup feed populate the region hub, hub and region.Regions are populated as follows

-   -   Region Id (From Lineup, Region Id)    -   Region Name (From Lineup, Region Name)    -   Division Id (From Lineup, Division Name)        Region Hub is populated as follows    -   Region Id (From Lineup, Region Id)    -   Hub Id (From Lineup, Hubs, it will be dissected into individual        records)        Hub is populated as follows    -   Hub Id (From Lineup, Hubs, it will be dissected into individual        records)    -   Hub Name (From Lineup, Hubs, it will be dissected into        individual records)        Downstream

Downstreams are described in greater detail below.

Records that were collected from the AS data files are stored in the SDAinstance 203 as collected. AS records are converted into the servicetable as defined in the Logical Data Model (LDM) and Data Dictionary.The LDM is a representation of the various data tables and therelationships between them.

XXI. Subscriber Reference Data—

Subscriber reference data files are received from a subscriber datasource 209. In one embodiment, the subscriber data source 209 comprisesa Convergys ICOMS server. In another embodiment, there is one instancefor the subscriber data collector 206.

Collection—

Protocol

The collection protocol is SFTP over TCP/IP link. Collection anddelivery of subscriber files is handled by the MSO, and pushed (SFTP) tothe collection system 200.

Compression

Files are not compressed.

Frequency

Files are, in one embodiment, delivered once per day.

File Naming Conventions

The file naming convention for reference data files is as follows:

-   -   <Div_ID>_<ref type>_YYYYMMDD.TXT        In the given naming convention, Div_ID is the division        identifier code, <ref type> is the reference type. The reference        type may be e.g., VIDEO_SUB or VIDEO_(—) SERVICES. The YYYYMMDD        symbolizes the year, month and day. The .TXT represents the file        extension. The following is an example of a data filename:        B411_VIDEO_SUB_20090211. TXT        Filename Duplication Check

Filename level duplication check is required. If the filename beingcollected already has been collected within a configurable period oftime then the file is considered as duplicate. In case of duplicate filefound an error message is generated and the file is stored to aduplicate file directory and processing continues with the next file.

The period of time for duplicate check is a parameter and isconfigurable. In one embodiment, for initial configuration thecollection system 200 is configured to check in the range of 1 to 5days.

Collection Process Steps

Once the system 200 collects the files, usage files are moved to localcollector directory and usage files are checked for duplicates.

Input Data Format

The input data format is pipe separated variables (PSV). The filescontain fields mapping to the table structures defined below.

Archiving

The system 200 maintains files for 7 days in one embodiment. They arestored into a dedicated directory on the local file system.

Business Logic—

Decoding

The collection system 200 decodes the records based on the schemadescribed below. Subscriber feed has the day a subscriber was added orremoved from service. The system 200 tracks those by using the connectdate. If a subscriber is not available in the feed, it is assumed thatit got disconnected from service. If a subscriber is disconnected fromservice all its equipment is assumed to be disconnected at the samedate. If equipment is not available in the feed, it is assumed that itwas disconnected from service. If a new equipment or subscriber is addedto the feed, it is assumed that it was connected to service in theconnect date that was delivered in the feed.

Validation Logic

The system 200 validates the collected records according to the schemadescribed in below.

Duplication Check Logic

The system 200 provides the ability to verify that subscriber recordsare not a duplicate of any other raw usage records.

Mapping Logic

The collection system 200 stages the collected records according to thesubscriber files schema described below.

Personally identifiable information (PII) identifying data that is usedfor correlation inside the data collection system 200 is, in oneembodiment, not anonymized. This data is considered reference data anddoes not reflect any customer usage. However, subscriber data isanonymized upon delivery to the SDA instance 203 by using theanonymizing algorithm described below, or other anonymization processes,such as the cryptographic hash described in previously incorporated U.S.patent application Ser. No. 11/186,452 filed Jul. 20, 2005 and entitled“Method and Apparatus for Boundary-Based Network Operation”.

The subscriber feed maps to the following fields for a subscriber:

-   -   Site ID    -   Division    -   Division ID    -   Customer Type    -   Subscriber Account (Anonymized)    -   Subscriber Zip Code    -   Subscriber Zip+4    -   HSD Household    -   Subscriber Connect Date    -   Video Services Household    -   Voice Service Household    -   Wireless Service Household    -   Company    -   Division Code    -   Franchise        The subscriber feed maps to the following fields for a service:    -   Customer Guid (Anonymized)    -   Site ID    -   Division    -   Division ID    -   Subscriber Account    -   Service Category Code    -   Service Occurrence    -   Service Code    -   Service Description    -   Service Connect Date    -   Mac Address    -   STB Serial Number    -   Customer Channel Lineup Id    -   STE Vendor    -   STB Model Number    -   STB Description    -   Equipment Connect Date    -   Cable Card Vendor    -   Cable Card Id    -   Cable Card Name        Downstream

Downstreams are described in greater detail subsequently herein.

Records that were collected from the subscriber files are stored in theSDA instance 203 as collected after all records have been anonymized byusing the algorithm described below.

Subscriber records are converted into the customer table as definedelsewhere herein.

XXII. Subscriber Reference Data—

FIG. 27 illustrates a matrix of the network elements and the downstream.As shown, the crossing cell of the network element and downstreamrepresents the action to be performed by the collection system 200. Theactions are as follows:

-   -   Yes The data from the network element is populated to the        downstream    -   No The data from the network element is not populated to the        downstream    -   Corr. The data from the network element is correlated and the        correlation result is populated to the downstream    -   P1 The data from the network element is populated to downstream        in phase 1    -   P2 The data from the network element is populated to downstream        in phase 2        Downstream Requirements—

The above requirements do not provide a product specific output. Rather,they provide a framework of general output requirements that productsmay rely on. The collection system 200 incorporates the ability toprovide output via SFTP of delimited files in pre-defined formats. Thesystem 200 also incorporates the ability to add pre-defined formats foroutput based on mutually agreed upon characteristics, including (i)appropriate query against data in the data dictionary, (ii) outputformat (comma delimited, pipe delimited, web-portal, etc.), and (iii)output frequency.

SDA—

In one embodiment, the SDA 203 comprises an Openet/Accenture subscriberdata analysis system. In another variant, there is one (1) instance ofthe SDA 203.

Output Data Format—

In one embodiment, SDA 203 is an independent instance that is co-locatedwith collection system 200. In another embodiment, the SDA system 203 ismoved from the collection system 200 when the solution gets deployed inproduction.

The output data format maps to the SDA logical data module for thefollowing inputs:

-   -   Services—Populated with data from AS data collector 210 and EPG        data collector 214    -   Customer and STB Household—Populated with anonymized data from        the subscriber table.    -   SDV Record—Populated with the correlated usage from tuner and        program events.    -   Advertisement Spot Play—Populated with the data from        advertisement data collector 212.    -   Video on Demand—Populated with from VOD data collector 208 after        the data has been enriched with subscriber information that is        anonymized.

The output schema matches the mapping schema described in the inputsection. Reference data that is only used for SDA 203 is inserteddirectly into SDA 203 and not managed by the collection system 200.

Delivery—

Protocol

Records that are destined to the SDA instance 203 are inserted usingSQL*NET in one embodiment.

Frequency

The delivery frequency is configurable but in one embodiment, isdefaulted to once every hour. If during technical specification it isdecided to co-insert both records or use Oracle stream this may be donein real time and the frequency does not apply.

Delivery Process Steps

The collection system 200 processes, filters, enriches and maps usagerecords as described above. After all records have been processed, theyare directly inserted into the SDA instance 203 (such as throughSQL*NET) either directly or through, in one embodiment, an Oracle toOracle connection. The following usage is propagated to SDA 203 fromcollected records.

-   -   Anonymized Subscriber Information    -   Tuning events that have been enriched with the programming and        subscriber data and corresponds to a viewing event.    -   Program Schedule    -   Video on Demand    -   Programming Data    -   Advertisement Spot Data    -   Master Channel information (Services Table)        Archiving

Records delivered to SDA 203 are archived in SDA 203. Records in SDA 203are retained, in one embodiment, for 90 days.

XXIII. National Cable Communications—

In one embodiment, there is one (1) instance of the NCC feed deliveringdata to the NCC.

Output Data Format—

In one variant, files are created in ASCII, Pipe Separated Value files(PSP). There are four files created for the NCC feed. These are (i) auniverse data file, (ii) a household data file, (iii) a station datafile, and (iv) a tuning data file.

Delivery—

Protocol

NCC Files are delivered using secured FTP (SFTP).

Frequency

The delivery frequency depends on the feed as follows:

-   -   Universe Data—Once a month    -   Household Data—Once a month    -   Station Data—Once a month    -   Tuning Data—Once a month        File Naming Conventions-Universe Data

The file naming convention for the universe data files is as follows:

-   -   Universe_<division>_<year>_<month>.dat        In the above, Universe_ is a literal, <division> represents the        division name, <year> is the creation year in four digits        (YYYY), <month> is the creation month in two digits (zero        padded, MM). The .dat symbol is a literal.

The file naming convention for the household data files is as follows:

-   -   Household_<division>_<year>_<month>.dat in which        According to the given convention, Household_(—) is a literal,        <division> is the division name, <year> is the four-digit        creation year (YYYY), <month> is the two digit creation month        (zero padded, (MM), and .dat is a literal

The file naming convention for the station data files is as follows:

-   -   Station_<division>_<region>_<year>_<month>.dat in which        The Station_(—) symbol is a literal, <division> represents the        division name, and <region> is the region name. The region name        corresponds to the lineup being used to map the channel id. The        <year> symbol represents the creation year in four digits        (YYYY), <month> is the creation month in two digits (MM) and        .dat is a literal.

The tuning data file naming convention is as follows:

-   -   Tune_<division>_<year>_<month>_<day>.dat in which        In the tuning data file naming convention, the Tune symbol is a        literal, <division> is the division name, <year> is the creation        year (four digits, YYYY), <month> is the creation month (two        digits, zero padded, MM), <day> is the creation day (two digits,        zero padded DD), and .dat is a literal.        Delivery Process Steps

The collection system 200 delivers the files to its destination viasecure FTP (SFTP).

Archiving

Universe Data, Household Data and Station Data are, in one embodiment,archived for 7 days. If Tuning Data are created on a daily basis, anddelivered at the end of the month, they are retained through the monthuntil delivery day. After the files have been delivered, they arearchived (in one example, for 7 days).

XXIV. Raw Record Store—

All records are stored in the raw record store before any validation orduplicate detection is applied. It serves as a reference fortroubleshooting purposes where output data records or any record hasbeen dropped from the processing stream (validation, duplicate detectionor filtering) is matched to its (original) version.

In one embodiment, one or more of the following requirements areimplemented: (i) retention policies dictate that all records remainavailable 7 days, and/or (ii) raw records are assumed to be “interimrecords”.

XXV. System Administration—

In one embodiment, existing system administration facilities are used.

XXVI. Alarming—

The collection system 200 may provide the ability to log an error ifconnectivity is lost between elements in the collection system 200. Theability to log an error if data received from a given source does notcomply with the expected format (Data Integrity Error) is also providedby the system 200.

XXVII. Reliability—

In one embodiment, the aforementioned collection system 200 and SDA 203do not follow production reliability metrics.

XXVIII. Extensibility Requirements—

The collection system 200 and/or SDA 203 may be configured to be able tobe expanded to do one or more of the following:

-   -   Increase the number of feeds being managed by the collection        system 200/SDA 203    -   Add the ability to add other types of feeds to the collection        system 200/SDA 203    -   Increase the number of reports in SDA 203    -   Increase the number of feeds from downstream    -   Change the number of regions being serviced        XXIX. Security Requirements—

Existing security facilities may be used for user and data control inone embodiment.

The collection system 200 ensures that collected turning records havebeen made anonymous. The collection system 200 also ensures thatcollected VOD records have been made anonymous.

XXX. Operating Environment—

Mediation Environment—

The mediation environment in one embodiment may include e.g., aproduction environment and a test environment. The productionenvironment comprises two dedicated servers (such as those illustratedin FIG. 20). The first server comprises the various software andhardware components which collectively comprise the data collectionsystem 200. The second server comprises at least two functions: (a) amanagement function for managing the system 200 running on the firstserver; and an analysis function (or SDA) 203 which analyzes collecteddata. The test environment is used to test patches, upgrades of theproduct or configuration changes and for troubleshooting productionproblems.

Hardware Requirements—

In one embodiment, the hardware platform is Sun™.

XXXI. Re-Processing—

In one embodiment, the system 200 provides the ability to locally retaintuning or VOD in any state between origination and landing in a datawarehouse. In one variant, the data is kept for up to (7) days.

Log Reprocessing—

Records that were collected from the SDV log are used to recreate thelost records for the period. The collected records are compared withother records that were collected prior to the outage. This is needed tocreate a correlation between the log UID and the geographicalinformation that was successfully correlated prior to the outage.

Using the established correlation, the subscriber information ispopulated for all the records that were lost during the outage. Insertthe viewing impressions records in the table that is used to feed theSDA 203 data warehouse. If any record has more than two ID's that arecorrelated, one before and one after the outage, the one that matchedafter the outage is selected. If any log record does not have a matchingTCP record, it is not rebuilt. Any viewing impression that was notterminated is terminated as an inactive session as part of reprocessing.An exemplary SDV log is illustrated in the table of Appendix DN ofpreviously incorporated U.S. Provisional Patent Application Ser. No.61/260,346.

After the process begins, the user identification (UID) in the logrecord is compared to the user identification (UID) in the TCP record toestablish a basis; an example is illustrated in Appendix DO ofpreviously incorporated U.S. Provisional Patent Application Ser. No.61/260,346. The process reconstructs the records that were lost duringthe outage with the comparison basis. Exemplary reconstruction isillustrated in Appendix DP thereof as well.

SDV Filtered Record Reprocessing—

SDV Records that were filtered because of duration may be reprocessed ifthe duration parameters are changed to a smaller duration. However, inone embodiment, only those records that were successfully correlated areeligible for reprocessing.

Records are selected by time period. The records that correspond to thetime period have their duration compared with a configurable filterduration parameter. Records in which its duration is larger than thefilter parameter are stored for delivery to SDA 203; otherwise theyremain as filtered.

XXXII. Sizing Requirements—

Exemplary sizing parameters are listed in the table of Appendix DQ ofpreviously incorporated U.S. Provisional Patent Application Ser. No.61/260,346 and are provided for sizing determination. In one embodiment,one or more of the following assumptions are taken into consideration:

-   -   1. Files are archived for 7 days.    -   2. Summarized data is archived for 7 days.    -   3. Operation tables are kept for 7 days.    -   4. Reporting tables are kept for 7 days.    -   5. Data sizing for the analytics database instance is        essentially the same as the one used for the mediation server.    -   6. Disk utilization is expected to have a high water mark of        80%.    -   7. Data for SDA 203 is located in the same server as the        collection management system 2006.        XXXIII. High Availability Requirements—

In one implementation, the collection system 200/SDA 203 is deployed ina high available environment.

XXXIV. Archiving Requirements—

The collection system 200 maintains files for 7 days in one embodiment.The files are stored into a dedicated directory on the local filesystem.

XXXV. Archiving Requirements—

The system 200 may provide the ability to log each action that a usertakes, including date, time, action details, and success indication.Further, the ability to log each action that the collection system 200takes, independent of user action, including date, time, action detail,reason and success may be provided as well. The collection system 200facilities may be used for the creation and management of logs.

XXXVI. Reporting Requirements—

Obtain Second-by-Second View of Household Viewership—

The second-by-second view of household viewership includes a tabularreport. In one embodiment, the report includes, at a minimum, the fieldsillustrated in Appendix DR of previously incorporated U.S. ProvisionalPatent Application Ser. No. 61/260,346.

In order to provide this view, the system 200 completes the followingcorrelation:

-   -   1) Tuning records are obfuscated and correlated to anonymous        households    -   2) Anonymous tuning records are correlated (via Source ID) to        video services as defined on the application server (AS)    -   3) Based on anonymous tuning record time and service,        programming information is mapped to tuning records        Obtain Second-by-Second View of Household Commercial Viewership—

The second-by-second view of household commercial viewership includes atabular report. In one embodiment, the report includes, at a minimum,the fields illustrated in Appendix DS of previously incorporated U.S.Provisional Patent Application Ser. No. 61/260,346. The reporting viewalso includes the ability to play the advertisement(s).

In order to provide this view, the system 200 completes the followingcorrelation:

-   -   1) Tuning records are anonymized and correlated to anonymous        households    -   2) Anonymous tuning records are correlated (via Source ID) to        Video Services as defined on the AS    -   3) Based on anonymous tuning record time and service,        programming information is mapped to tuning record    -   4) Match Ad Zone to SDV Service Group/SDV Server    -   5) Match Ad Zone “program identity” with Services, as defined in        the AS    -   6) Match Ad Spot play (from Verification File) with network and        program information (based on mappings in 4 and 5 above)    -   7) Match Ad Spot ID to Advertiser        Obtain View of VOD Consumption—

The view of VOD consumption includes a tabular report. In oneembodiment, the report includes, at a minimum, the fields illustrated inAppendix DT of previously incorporated U.S. Provisional PatentApplication Ser. No. 61/260,346.

In order to provide this view, the system 200 completes the followingcorrelation:

-   -   1) VOD records correlated to anonymous households        XXXVII. Error Management—

In one embodiment, the collection system 200 facilities are used forerror management.

XXXVI. SDA Data Warehouse—

After mediation, information comes to rest in SDA 203 the datawarehouse. The data listed below is hosted in the data warehouse andmanaged outside the mediation scope. This data is inserted usingdatabase tools or by hand. In the instance the data is entered by hand,the data comes through a spreadsheet. The following data is hosted inthe data warehouse for analysis:

-   -   a. Subscriber Region [1-DC1]    -   b. Media Sales Region [1-DC2]    -   c. Division [1-DC3]    -   d. DMA [1-DC4]    -   e. DMA Zip [1-DC5]    -   f. Narrow Cast Area [1-DC6]    -   g. Narrow Cast Types [1-DC7]    -   h. Household [1-DC25]    -   i. Household Packages [1-DC26]    -   j. Household Demos [1-DC27]    -   k. Household Demos Range [1-DC28]    -   L. STB Household [1-DC29]    -   m. Household STB EPG [1-DC30]    -   n. Household STB Agents [1-DC31]    -   o. Household Opt-In Data [1-DC33]    -   p. Household Opt-In Phone [1-DC34]    -   q. Household Opt-In Email [1-DC35]    -   r. Household Contact Type [1-DC36]    -   s. Household Hub Information [1-DC37]    -   t. SDV Server Config [1-DC38]    -   u. SDV Server Service Group [1-DC39]    -   v. VOD Server Config [1-DC40]    -   w. VOD Server Svc Group [1-DC41]    -   x. Ad Zone [1-DC42]    -   y. Ad Zone Zip Code [1-DC43]    -   z. Sources [1-DC44]    -   aa. Packages [1-DC45]    -   bb. Channel Map [1-DC46]    -   cc. Lineup [1-DC47]    -   dd. Service [1-DC48]    -   ee. Service Type [1-DC49]    -   ff. Program [1-DC50]    -   gg. Rating Code [1-DC51]    -   hh. Viewing Type [1-DC52]    -   ii. Program Genre [1-DC53]    -   jj. Genre [1-DC54]    -   kk. Program Event Flag [1-DC55]    -   ll. Event Flag [1-DC56]    -   mm. Program Adv Rating [1-DC57]    -   nn. Advanced Rating [1-DC58]    -   oo. Tuning Event [1-DC59]    -   pp. Tuning Event Types [1-DC60]    -   qq. VOD Assets [1-DC67]    -   rr. VOD Asset Metadata [1-DC68]    -   ss. VOD Event Type [1-DC70]    -   tt. VOD Stop Reason [1-DC71]    -   uu. VOD Segments [1-DC72]    -   vv. VOD Segment Type [1-DC73]    -   ww. VOD Trick Plays [1-DC74]    -   xx. VOD Trick Type [1-DC75]    -   yy. Local Ads [1-DC81]    -   zz. Advertiser [1-DC82]    -   aaa. Advertiser Type [1-DC83]    -   bbb. Agency [1-DC84]    -   ccc. Ad Status Code [1-DC86]    -   ddd. CPE [1-DC9]    -   eee. HD Resolutions [1-DC10]    -   fff. Cable Card [1-DC11]

The format of the SDA 203 is described below. Of the previous list, thefollowing data items are, in one embodiment, delivered throughmediation:

-   -   a. Tuning Event [1-DC59]    -   b. Sources [1-DC44]    -   c. Packages [1-DC45]    -   d. Channel Map [1-DC46]    -   e. Ad Spot [1-DC85]    -   f. VOD Event [1-DC69]        XXXVII. Assumptions—        General Assumptions—

All events are collected in the network elements' local time. Some ofthese network elements do collect data in UTC. All timestamps arenormalized to UTC in one embodiment.

Events from an ETAG server may be collected and managed.

Inspection of content of High Speed Data requires a content inspectiondevice which is, in one embodiment, placed in network.

In one embodiment, online data is retained for a period of 13 months andoffline data is retained for a period of 18 months in archive. Theretention period for all data in the SDA system 203 may be of 7 days inthe collection system 200 and 90 days in the SDA instance 203.

A unifying identifier for all network services is provided by the AS. Anidentifier that maps the advertisement server head ends to specificdemographic locations is provided by DNCS.

All direct database usage insertion from foreign systems may done on anhourly basis in one embodiment. Viewing impressions as collected fromthe SDV server 207 in one embodiment follow the transitions described inthe state diagram illustrated at FIGS. 24 and/or 25. This may alter theway that viewing impressions are computed for multi-tuner environments,as discussed elsewhere herein.

SDV—

TCP connection does not require a bidirectional protocol. The TCPprotocol echoes what is written in the log with the difference that theuser ID being written at the log is already anonymized while the onebeing delivered through the TCP feed is the actual one.

SDV programming data may be correlated with the Service Id through theSource ID that is being delivered through the AS feed. If this is notpossible, a translation table is built.

In embodiment, DVR playback events are not collected through the SDV,and only DVR recording events are collected.

In one embodiment, the TCP connection feed is in ASCII, CSV. If the TCPconnection gets severed, the data is not buffered and is consideredlost.

Traffic and Billing Feed for Ad's Play—

The network name is delivered in a way that can be enriched with a validservice Id. If this is not unique, the network name is enriched with thedivision ID to obtain a unique service Id. In one embodiment, a trafficand billing (T&B) entity is used to deliver data from the advertisementdata source 215. The data from the T&B is written in lieu of theadvertisement data collector 212 feed.

Subscriber Feed—

The subscriber feed may, in one embodiment, utilize methods similar tothose used for digital phone and high speed data solutions. Thesubscriber feed data may comprise ASCII, Pipe Separated Value datadelivered through FTP.

XXXVII. Data Formats—

SDV Data Formats, Input Format—

The SDV collection format is specified elsewhere herein.

VOD Data Replication, Input Format—

In one embodiment, the VOD collector comprises an Everstream VODcollector. According to this embodiment, the following tables will bereplicated using the schema described in the Everstream Data Dictionary.

-   -   LOCATION    -   ORGANIZATION    -   EQUIPMENT    -   VOD_LEASE    -   VOD_STREAM    -   VOD_ASSET    -   VOD_ASSET_TITLE    -   VOD_ASSET_TITLE_GENRE    -   VOD_ASSET_COMP    -   CALENDAR    -   TIME_OF_DAY        VOD Asset SDA Delivery Format, Output Format—

VOD usage data is delivered to SDA 203 in a database table such as thatillustrated in Appendix DU of previously incorporated U.S. ProvisionalPatent Application Ser. No. 61/260,346.

MAS—TWC Generated, Input Format—

Data from the AS is ASCII in which its records are separated using pipe(|).

Channel File

For each LineUpGroup/region, a display channel list is produced by theDigital Navigator (e.g., SARA, Passport, Mystro). An exemplary displaychannel list definition is shown in Appendix DV of previouslyincorporated U.S. Provisional Patent Application Ser. No. 61/260,346.

Services File

A service list is produced by the Digital Navigator (e.g., SARA,Passport, Mystro). All services are included, however for services whichhave a single parameter containing a sourceID (i.e., “watchtv”) thisparameter is put into the sourceID field, otherwise it is omitted. Anexemplary service list definition is provided in Appendix DW ofpreviously incorporated U.S. Provisional Patent Application Ser. No.61/260,346.

Lineup File

The relationship of Lineup is produced by the Digital Navigator (e.g.,SARA, Passport, Mystro). An exemplary Lineup definition is provided inAppendix DX of previously incorporated U.S. Provisional PatentApplication Ser. No. 61/260,346.

EPG Data, Input Format—

In one embodiment, EPG data is delivered in compressed format usinggzip. Uncompressed, the files are formatted in ASCII. The EPG datafields may have variable length and be pipe (0x7C) separated. Recordsmay also be new line (0x0A) terminated. Date and time fields reflectCoordinated Universal Time (UTC).

Subscriber Detail Extract—

The subscriber detail extract contains the most granular information forsubscribers, accounts and equipment. The subscriber detail files maycontain fields mapping to the exemplary table of Appendix DY ofpreviously incorporated U.S. Provisional Patent Application Ser. No.61/260,346.

Subscriber Services Extract—

The subscriber services extract file contains the most granularinformation for equipment, services and packages. In one embodiment, thefile contains fields mapping to the table of Appendix DZ of previouslyincorporated U.S. Provisional Patent Application Ser. No. 61/260,346.

Advertisement Data Format—

Eclipse Open TV Data Replication

In one embodiment, advertisement data is collected via an Eclipseserver. According to this embodiment, the following tables arereplicated using the schema described in the Eclipse EDA DataDictionary:

-   -   INVOICE_DETAIL    -   ADVERTISER    -   CONTRACT    -   COPY    -   COPY_GROUP    -   REGION    -   HEADNET    -   RETAIL_UNIT    -   NETWORK    -   PROGRAMS    -   SPOT

Ad Insertion Events SDA Delivery Format

The table of Appendix EA of previously incorporated U.S. ProvisionalPatent Application Ser. No. 61/260,346 may, in one embodiment, be usedto insert the advertisement insertions from the advertisement datacollector 212 schema.

Ad Insertions Events Reference Data Fields

The table of Appendix EB of previously incorporated U.S. ProvisionalPatent Application Ser. No. 61/260,346 is, in one embodiment, merged bythe collection system 200 with the data that is provided by theadvertisement data collector 212. In one embodiment, the table is mergedevery day.

National Cable Companies, Output Format—

The delivery of records to the National Cable Companies (NCC) isspecified above with respect to the Audience Measurement Interface.

Exemplary Advertisement Data Format—

In one embodiment, the advertisement data comprises 17 fields defined asfollows:

-   -   Field #1—Event Type—Bytes 1-3.    -   Type of event defined by record    -   Additional acronyms to be determined.        Types—    -   LOI=Local Originated Insertion    -   REM=Remark, no action.    -   END=Last record, lone field record. (Optional)    -   NUL=Ignore Break Cue        Field #2—Scheduled Date—Bytes 5-8.        T&B's approximation of the date when the event will occur.    -   Format—MMDD    -   Ex. 1210        Field #3—Schedule Time—Bytes 10-15.        T&B's approximation of the time of day when the event will        occur.    -   Format—HHMMSS    -   Ex. 020345        Field #4—Window Start Time—Bytes 17-20.        Time of day to open window of opportunity for event to occur.    -   Format—HHMM    -   Ex. 1535        Field #5—Window Duration Time—Bytes 22-25.        Length of time of window of opportunity for event to occur.    -   Format—HHMM    -   Ex. 0100        Field #6—Break Number Within Window—Bytes 27-29.        Break sequence number within window of opportunity for event to        occur.    -   Ex. 002        Field #7—Position Number Within Break—Bytes 31-33.        Position sequence number for event within break.    -   Ex. 001        Field #8—Scheduled Length—Bytes 35-40.        Scheduled event (spot) length    -   Format—HHMMSS    -   Ex. 000030        Field #9—Actual Aired Time—Bytes 42-47.        Actual aired time of day.    -   Format HHMMSS    -   Ex. 012315        Field #10—Actual Aired Length—Bytes 49-56.        Actual aired length of spot    -   Format—HHMMSSCC    -   Ex. 00003000        Field #11—Actual Aired Position Within Break—Bytes 58-60.        Actual sequential position number that event occurred    -   Ex. 002        Field #12—Spot Identification—Bytes 62-72.        T&B's spot identification code.    -   EX. 00000132401        Field #13—Status Code—Bytes 74-77.        Completion Status Code    -   Ex. 0001        Using the following table of definitions.    -   0001=Aired Successfully    -   0002=Generic Failed to Air    -   0004=Failed, Bypass On    -   0005=Failed, Logging Turned Off    -   0006=Failed, Bad Video    -   0008=Failed, User Abort    -   0009=Failed, Inserter Abort    -   0010=Failed, Device Not Ready    -   0012=Failed, Unknown Error    -   0013=Failed, Time Out    -   0014=Failed, Device Time Out    -   0015=Failed, System Error    -   0016=Failed, Operator Error    -   0017=Failed, Inserter Busy    -   0018=Failed, Insertion Conflict    -   0019=Failed, Directory Error    -   0020=Failed, No Ad Copy in Inserter    -   0021=Failed, Cued Late    -   0022=Failed, Channel Collision    -   0023=Failed, No Cue in Window        Field #14—Advertiser Name—Bytes 79-110        Advertiser's name as identified in T&B. This is not a required        field and may be left out.        Field #15—Advertiser Spot Name—Bytes 112-131.        Advertiser's Spot Name as Identified in the T&B. This is not a        required field and may be left out,        Field #16—Scheduler Fill—Bytes 133-136.        Identifies the spot as either being scheduled contractually or        used as a filler in order to complete a commercial break. This        is not a required field and may be left out.

Options—

-   -   Sch=Scheduled    -   Fill=Filler spot        Field #17—User Defined—Bytes 138-NNN.        For use in tracking other data. This is not a required field and        may be left out.        XXXVIII. Anonymizing Algorithms—

Customer identifiers (e.g., MAC, Subscriber Accounts, Customer Accounts)may, in one embodiment, be made anonymous after being is correlated withgeneric demographic information prior to its delivery to the SDAinstance 203. The obfuscation of customer identifiers is done in thecollection system 200, after the guiding of subscriber information hasbeen completed.

Non obfuscated records may remain in the collection system 200 for thepurposes of auditing and reconstruction. Non obfuscated records may bedelivered to the SDA instance 203 in one instance, if they belong tocustomers that have “opt-in” as indicated in their provisioning record.

In one embodiment, the following fields are obfuscated as applicable:

-   -   MAC ID    -   Serial Numbers    -   Account Numbers    -   Telephone Numbers    -   Customer Identifiers

In another embodiment, the following fields are not obfuscated:

-   -   Correlated Zip Code    -   Correlated DMA    -   Correlated Division ID    -   Correlation Division Name

In one embodiment, the anonymization occurs by inter alia, the use of acryptographic hash as described in the previously incorporated U.S.patent application Ser. No. 11/186,452 filed Jul. 20, 2005 and entitled“Method and Apparatus for Boundary-Based Network Operation”.

Alternatively, customer accounts may be obfuscated via the followingalgorithm:

-   -   Collect the division ID (DI) from the billing system that was        correlated with the provisioning data.    -   Collect the Customer ID (MAC ID in this case) that was        correlated with the provisioning data.    -   Concatenate DI to the end of CI yielding UID (DI+CI=UID)    -   Using the Hexadecimal representation of the UID, divide the UID        in DWORDS from “right to left” being a DWORD defined as a double        word, 32 bits wide. Pad the least significant DWORD with zeroes        up to 32 bits.    -   From the first converted DWORD, add third salt value. Carry any        overflow bits to the adjacent DWORD, continue carrying if        needed.    -   Subtract fourth salt value from the second DWORD.    -   Continue this process, adding the first value to odd DWORDS,        subtracting the second value from the even DWORDS until all        DWORDS have been processed.    -   Convert the series of DWORDS back to its ASCII character        representation.    -   Perform an MD5 hash of the resultant string.    -   The output of this MD5 is the obfuscated Customer ID

MAC addresses may be then obfuscated.

Records collected from the SDV feeds may obfuscated using the followingalgorithm:

-   -   When a tuning record is collected the following fields may be        obfuscated:        -   MAC Address from record        -   Division ID from correlation with provisioning        -   Customer ID from correlation with provisioning    -   To the Division and Customer ID apply the Customer ID        obfuscation process    -   To the MAC address apply the MAC obfuscation process.    -   Replace the Division and MAC with its obfuscated counterparts.        Records collected from VOD are obfuscated, in one embodiment,        using the following algorithm:    -   When VOD record is collected, obfuscate the following fields:        -   MAC address from record        -   Division ID from correlation with provisioning        -   Customer ID from correlation with provisioning    -   To the Division and Customer ID apply the Customer ID        obfuscation process    -   To the MAC address apply the MAC obfuscation process.    -   Replace the Division and MAC with its obfuscated counterparts.        XXXIX. Logical Data Model—

The appended logical data model describes the way the input tables aremapped to the output in the SDA instance.

XL. Advertisement Insertion Events—

The SDA Audience Measurement platform 203 stores events relating to theconfirmation of linear advertisement insertions. Such events provide theMSO with the ability to identify which advertisements were inserted,when they were inserted, on which channel the advertisement was insertedand the number of households that the ad was available to. Secondly, byanalyzing this data alongside the SDA-AM 203 tuning events for thecustomers tuned in to that channel at the time of the advertisementplay-out, a calculation of the number of customers watching theadvertisement can be determined.

This document provides an overview of the business events which definethe various instances of valid advertisement insertion events.

Advertisement insertion events stored in the SDA-AM 203 are defined inthis document as being derived from logged events detailing the successor failure of a linear spot play-out confirmation received from anadvertisement insertion server. The confirmation message contains at theminimum the time, date, universal channel reference, and theadvertisement-asset description.

Ad Insertion—

Local advertisements are inserted into specific channels. Theseinsertions are typically performed via inserting over the top (i.e.using MPEG splicing technology) of existing ads in broadcast signals.The MSO may utilize MPEG advertisement insertion devices to spliceadvertisement-assets into broadcast signals using advertisement serversto provide the advertisement-asset.

A channel in which an advertisement may be inserted provides anindication of when one may be inserted through the use of a data trigger(i.e. provided alongside the audio and video content using the SCTE-35standard). The advertisement insertion system, on detecting the trigger,inserts an advertisement via the MPEG splicer—the advertisement isobtained from the advertisement-server, where the choice ofadvertisement asset is pre-determined according to the details of thevarious scheduled ad advertisement campaigns for that channel and timeperiod.

Ad Server Generated Insertion Events and Reference Data Sources—

Each advertisement insertion server generates a log of events describingeach advertisement insertion event. These events describe successful andfailed insertion attempts, as well as the type of event (i.e. normal,filler bonus, or make-good).

There are three primary objectives in storing advertisement insertionswithin the SDA 203:

-   -   Provide an indication of all successful advertisement playouts.    -   Provide reference data by which the advertisement insertion        events can be enriched for reporting purposes.    -   Provide the ability to determine how many customers viewed a        given advertisement.

While the first two objectives are mostly straightforward, determiningactual advertisement impressions is complicated by the fact that thesystems responsible for delivering video content and advertisementcontent use different representations for key data elements. A means ofnormalizing advertisement playout data so that it can becross-referenced against tuning data is therefore provided.Advertisement event attributes are not dependent on any particularunderlying technology (e.g., are system agnostic), as the MSO mayutilize different advertisement insertion solutions.

Ad Playout Information—

The advertisement event attributes associated with a playout of anadvertisement may be similar to those listed in Appendix EC ofpreviously incorporated U.S. Provisional Patent Application Ser. No.61/260,346.

Ad Reference Information—

A set of reference data which may be used by the mediation componentwithin the SDA 203 to enrich the advertisement events may be utilized asprovided in the example of Appendix ED of previously incorporated U.S.Provisional Patent Application Ser. No. 61/260,346.

Ad Normalization Information—

Appendix EE, of previously incorporated U.S. Provisional PatentApplication Ser. No. 61/260,346, may be utilized to provide an abilityto normalize an advertisement against tuning events to determine actualadvertisement impressions. The data needed to perform this normalizationrelates to both 1) network and 2) location information.

Ad Event Normalization—

Specific terminology is used to describe the regions in which local adsare sold. Syscodes describe zones in which local advertising can besold. Syscodes represent areas with geographic proximity and oftendemographic consistency. Syscodes often represent a collection of zipcodes that change over time due to various advertisement industrydrivers. While Syscodes are used in the advertisement industry tomaximize advertisement sales effectiveness, other naming standards fordefining video regions may be used. In one embodiment, customers receiveservice packages represented within a channel lineup, which is served tohubs located at geographically-strategic locations based on bandwidthand capacity considerations. In mapping customer viewership (tuningevents) to advertisement sales (Advertisement Events), correlationbetween the video Hubs and Syscodes is used.

Once the data is acquired, processing is performed in two main areas:(i) geographic alignment and (ii) service alignment. Geographicalignment comprises steps to determine the exact geographic region theadvertisement actually played in. Advertisement insertion servers arelocated at the headend, thus a conversion is necessary into a set ofhubs and nodes which relates to the geographic reference in which thetuning events are stored. Service alignment comprises steps necessary toconfirm the channel on which the ad was inserted. Multiple versions of aparticular channel are present at the headend (i.e. insertionpoint)—these channels are sent across the served geographic region, andthe channel instance used at each node/hub. The details of eachprocessing step are provided below.

Geographic Alignment—

Geographic Alignment processing steps aim to identify where the channelin which the advertisement was inserted and/or was distributed. When anadvertisement playout occurs, the only geographic information associatedwith the spot is a retail unit code.

From the retail unit code (e.g. CMC) provided in the advertisementplayout, the Syscode(s) can be determined via reference data and storedin SDA 203.

From the advertisement Syscode(s), a collection of zip codes associatedwith the Syscode(s) can be determined. In one embodiment, thisinformation is maintained in the CableTrack system, and verified foraccuracy. The Syscode-zip code mapping is maintained in SDA 203.

In some instances, a Syscode does not align directly with a zip code(e.g. the zip code is split across two Syscodes). When a zip code ismapped to multiple Syscodes, the entire zip code is considered as withinthe Syscode of a given advertisement. This may lead to someover-counting of ad impressions, but this double-mapping occursinfrequently.

From the retail unit code, the Division ID (e.g. CLT) is associated withthe advertisement playout. A Region represents a collection of relatedSyscodes. While the media sales-based region and video-based divisionare expected to align, a mapping is maintained within SDA 203 to ensureconsistency.

The following information is written to the SDA 203 advertisement event:Division ID, Retail Unit Code.

The geography mapping table within SDA 203 may in one embodiment containthe fields listed in Appendix EF of previously incorporated U.S.Provisional Patent Application Ser. No. 61/260,346. This data is used inthe advertisement impression reporting process discussed below.

Service Alignment—

Service Alignment is the process by which service-level information fromadvertisement playout records are correlated to standard video services.When an advertisement playout occurs, the only service informationassociated with the Spot is a network code.

From the network code (e.g. CNN-HD) in an advertisement playout, analignment is performed to associate the network on which theadvertisement aired to tuning events. The network code is a free-text‘friendly’ identifier for a network within a region. While these networkcodes are often easily identifiable, they are not standardized and thusare not a reliable source for tuning event correlation. For example,CNN-HD may be referred to as CNN_HD, CNN High Def, CNN-HDTV, etc. Tuningevents (as defined below) are normalized on unique Service ID from theApplication Server (AS). In order to associate advertisement events totuning events, the advertisement event is also normalized on the ASService ID.

A Service ID-to-network code reference table is maintained by the SDA203. This table maps a network code to a unique AS Service ID withineach Division. There are certain occasions where a network code may mapto multiple Service IDs within the same Division. This mostly occurs onlocal channels such as news or weather, where the channel could differat the county level. In these cases, an additional retail unit field maybe used to define a single Service ID for networks defined atsub-Division level.

The following information may be written to the SDA 203 advertisementevent: Service ID.

The service mapping table within SDA 203 may in one embodiment containthe fields illustrated in Appendix EG of previously incorporated U.S.Provisional Patent Application Ser. No. 61/260,346. This data is used inthe advertisement impression reporting process discussed below.

SDA-AM Ad-Insertion Events—

An advertisement campaign is defined as a single contract containing oneor more advertisement assets (or Copys) which are inserted at specifictimes and on specific channels.

A Spot is defined as a specific insertion opportunity (indicated withthe use of triggers on channels) on a specific channel. Generally, 2minutes worth of advertising inventory are available per hour on each ofthe channels in which an advertisement insertion agreement exists. Thisadvertising inventory is split up typically into four 30-second spots.

An advertisement insertion event is generated when an advertisementserver inserts an advertisement (from a specific advertisement campaign)using an MPEG splicer on a linear channel after receiving a validSCTE-35 trigger event generated on that particular channel.

Advertisements which are inserted according to the terms of the campaigncontract are considered normal advertisement types.

Advertisement insertion events are generated on each insertion andrelate to insertions occurring on a single channel and consist of asingle advertisement asset.

Advertisement campaigns are typically owned by a particular advertiser,and purchased from an MSO by a particular advertising sales agency, andbrokered using a particular account executive.

Regions in which advertisements are inserted are called Syscodes—eachhousehold in a Syscode receives the same linear channel, and hence thesame advertisement if inserted on a channel for that Syscode.

Ad Impression Report Generation

The diagram of FIG. 28 provides an example of how an advertisementinsertion event is used to determine viewership when storingadvertisement insertion events as defined herein.

Ad Event Creation—

The AD_INVOICE table (e.g., OpenTV's INVOICE_(—) DETAIL) is obtained ona daily basis. The AD_INVOICE table contains key fields relating to theadvertisements creative, network, and geographic location to where theadvertisement was displayed.

The advertisement's retail unit is used to query the RETAIL_UNIT tableto determine the collection of Syscodes associated with the Retail Unitwhere the advertisement was played.

The SDA's 203 GEOGRAPHY_MAPPING table may be used to determine theDivision where the advertisement was played using market, retail unit,or Syscode information. The Division ID is written to the advertisementevent.

After the Division ID is determined, the advertisement's network code isused to query the SDA's 203 SERVICE_MAPPING to determine the Service IDassociated with that network code in the specified Division. The ServiceID and Service Name are written to the advertisement event. Ifnecessary, the retail unit code may be used to identify a Service IDwhere there might be multiple versions of a network (e.g. for News orWeather broadcasts)

The advertisement event's end rime can be deduced from the air time,date, and copy length information stored with the AD_INVOICE. Additionalattributes are stored on the advertisement event fordimensioning/reporting purposes as well.

Ad Report Generation—

A query is made to determine how many customers watched a specificadvertisement based on an advertisement identification (Ad ID).

The advertisement event associated with the Ad ID is retrieved from SDA203. The Service ID, date, start time, and end time of the advertisementevent are correlated against tuning events to determine how many tuningevents exist which meet the following characteristics:

-   -   The Service ID of the tuning event matches the advertisement        event's Service ID    -   The start/end times of the tuning event indicate that the tuning        event has at least a one-second overlap with the advertisement        event [this exact definition can be configured by different        reporting mechanisms based on business requirements].

The advertisement event's retail unit is used to determine the Syscodeswhere the advertisement was displayed. SDA's 203 GEOGRAPHY_MAPPING tableis used to determine the collection of zip codes associated with theSyscode(s).

Each tuning event has a customer GUID which uniquely identifies thecustomer who generated the tuning event. A cross-reference to theCUSTOMER table indicates which tuning events were generated in thecollection of zip codes where the advertisement event was generated.

XLI. SDA Interactive Application Events—

Interactive applications introduce additional functionality totelevision viewers through the use of on-screen applications. Generally,there are two broad categories of interactive applications:

-   -   Bound applications: tied to television content and only        displayed on the television or made available for invocation by        the customer when explicitly triggered by the content owner    -   Unbound applications: stand-alone applications which are        agnostic to the programming being shown by the content owner.        These are typically invoked through user action—i.e. invoked by        the user through the on-screen menu or via a specific short-cut        button on the remote control unit.

Interactive applications provide benefits to the MSO via the developmentof a differentiated video service offering to enhance customer'sexperiences allowing a multitude of interactive capabilities (i.e.advertising, promotions, games, etc). In addition interactiveapplications allow access to third party providers to host applicationsavailable through the television—while keeping the customer's focus onthe television.

In order to fully capitalize on the delivery and use by customers ofinteractive applications, the collection system 200/SDA 203 ensureappropriate measurement of actual and potential reach of theapplications, customer engagement with applications and on the successof applications.

Interactive application capable CPE 103 and regions are tracked withinthe SDA 203 as reference data, in addition to details around eachinteractive application (type of application, format and rating ofapplication).

Customer usage of interactive applications is stored as interactiveapplication events, which are analyzed for customer experience,marketing and advertisement sales revenue calculation purposes.

In order to ensure that the customer use of interactive application isappropriately captured for measurement purposes, it is important toensure that the data collection methods, the events being captured, andusage metrics derived from the raw data are consistently defined, andthat the manner in which metrics are stored are sufficient to supportthe business needs of the measurement.

An interactive application may exist in various states with respect touser engagement measurement

Valid interactive application events may be stored within the SDA datastore and subsequently used to create business metrics.

Interactive Applications Definition—

An interactive application is defined as a program which runs on a CPE103 (and displayed on a television) when invoked by a customer. Theinteractive application can be called from a user menu (unbound) orinvoked during a particular program or advertisement (bound).

Interactive application delivery involves a number of platform elements:

-   -   Interactive Application Streamer: provides a carousel in which        the interactive application program is repeatedly transmitted        and subsequently inserted in-band on the appropriate video        multiplex.    -   MPEG-2 Transport Stream Multiplex: Equipment which sits in the        path of the video signal, in which the interactive application        is received from the Streamer and inserted into a downstream        signal to the STB.    -   Interactive Application trigger: a SCTE-130 compliant message        sent alongside the video signal to indicate when an interactive        application prompt should be displayed to the viewer by the        Interactive Application client—used for bound applications.    -   Interactive Application Client (User Agent): Program code which        resides on the CPE, and performs two actions: (1) listens for        interactive application triggers, (2) renders interactive        application programs onto the television set once invoked,        either by the trigger for bound apps, or by the user, through        interaction with digital navigation software, for unbound apps.    -   CPE: the device on which the Interactive Application Client        resides, and the platfotin used to render and run interactive        applications once invoked.    -   Return Path: IP return path, in SA plants, currently enabled        through the DAVIC protocol between the CPE and appropriate Demod        card at the headend. In one embodiment, the return path is        enabled through the Motorola ALOHA protocol between the CPE and        appropriate return path demodulator (RPD) at the headend. In        another embodiment the so-called Tru2Way path is used, this        return path is enabled by the DOCSIS modem residing within        Tru2Way CPE establishing a communication path with the Cable        Modem Termination System (CMTS). This IP return path is used by        the interactive application program to relay messages and        commands to and from to the Interactive Application Server.    -   Interactive Application Server: performs server functions for        interactive applications running on the CPE—typically, these are        similar to web-servers and can provide data to applications if        requested, and acts as the destination for interactive        application events which indicate viewer engagement.

FIG. 29 describes how these platform elements are utilized to deliverinteractive applications (iApps).

As illustrated in FIG. 29, an interactive application (iApp) ismultiplexed with video content and invoked upon a request from a userCPE. This is accomplished by the following steps:

iApps are managed through a campaign manager, which confirmsavailability of bandwidth to stream the application.

iApp owner provides iApp reference data (e.g. metadata) and iApp contentto an application management layer.

The application management system provisions the iApp Server withdetails of the iApps and the iApp content management system with theiApp content (not shown).

iApp content management systems send interactive content to iAppstreamers for subsequent playout as instructed by the campaign details.

External content is typically included within the video stream (as inthe case with cable network provided interactivity) along withstandards-based triggers which are used to signal CPEs to invoke boundiApps on the user TV screen.

For interactive content which is stored internally iApp streamers insertiApp content for downstream transmission within linear broadcast stream.

iApp content is multiplexed with the video signals as MPEG-2 datapackets inside the MPEG-2 Transport Stream (in-band)

Compliant CPE contain, in one embodiment, an EBIF User Agent which caninvoke an iApp when initiated by customer (for unbound apps) or whentriggered within the video feed (for bound apps).

For bound applications, one or more “overlays” may be presented toviewers watching specific programming. Depending on business rules,targeting considerations, and subscriber preferences, some viewers maynot be presented with overlays even though they are viewing specificprogramming in which other viewers would be presented with an overlay

A user utilizes their remote control units to interact with theapplication, if necessary; alternatively the CPE may comprise one ormore buttons, a touch screen, etc. for interaction with the application.

The iApp, once invoked, utilizes the CPE return path to receive andtransmit application messages to the iApp Server based on customerinteraction.

The iApp Server, for some applications, aggregates specific userresponses (e.g. Votes) and submit results via an TAF message (i.e. backto app owner).

The iApp content management system provides iApp playoutconfirmation/details to an SDA mediation layer.

The iApp Server provides iApp Usage Data to SDA Mediation Layer usingIAM interface.

The iApp owner provides iApp reference data to SDA 203 via CTPinterface.

The SDA 203 mediation layer normalizes and enriches iApp reference andusage data into a collection of anonymized interactive events.

The SDA 203 may generate and provides iApp usage reports.

In one embodiment, the SDA 203 formats iApp usage report for specificcampaigns using the SMS format and submits them to authorized users viathe SMSI (e.g. Canoe).

The SDA 203 exposes Subscriber Information Service to SCTE-130 systemsin performing DAT, and or targeted advertising.

The SDA exposes Subscriber information, T-commerce, and other SOAservices for use by iApp servers to fulfill customer app functions(T-Commerce, Premium Upsell, callerID, preferences, credit limits, etc).

Standards & Conventions—

The exemplary naming conventions for the components discussed above arelisted in Appendix EH of previously incorporated U.S. Provisional PatentApplication Ser. No. 61/260,346 and may be implemented.

The scope of the SDA program focuses, in one embodiment, around EBIFapplications. These applications have been classified into one of threecategories (device classes), and are examples of each are summarized inthe table of Appendix EI of previously incorporated U.S. ProvisionalPatent Application Ser. No. 61/260,346. Each device class refers to thecomplexity of the type of EBIF application, in terms of STB computingpower and runtime memory resources required to execute the applicationfunctionality. Consequently each class of application requires a minimumCPE specification. Generally, a higher device class number (1, 2, or 3)correlates to a more complex/robust device.

As defined in the table of Appendix EI, three types of devices areinitially defined for Advanced Advertising Services. Any of the itemswithin a comma separated list may be present. An ampersand is used toindicate when multiple options must be present.

Exemplary CPE makes and models which may support EBIF applications (ifthey have the correct version of middleware platform installed), andtheir relative EBIF class classification is described in the table ofAppendix EJ of previously incorporated U.S. Provisional PatentApplication Ser. No. 61/260,346.

Interactive Application Examples—

The composition of an interactive application may consist of a number ofapplication classes which are presented to the customer as part of anapplication screen flow. Some examples of application classes that canexist, and any particular interactive application would potentiallycombine one or multiple classes depending on the creative requirementsof the application owner can be found in the table of Appendix EK ofpreviously incorporated U.S. Provisional Patent Application Ser. No.61/260,346.

An example interactive application screens, and describes their use ofthe interactive application classes can be found in the table ofAppendix EL of previously incorporated U.S. Provisional PatentApplication Ser. No. 61/260,346.

Additional interactive applications are discussed in co-owned,co-pending U.S. patent application Ser. No. 12/582,653 filed on Oct. 20,2009 and entitled “Methods and Apparatus for Enabling MediaFunctionality in a Content-Based Network”, which is incorporated hereinby reference in its entirety.

Customer Opt-Out—

Interactive applications may be considered an enhancement to basicservices, and as such, in one embodiment a customer may opt-out ofreceiving these enhanced services. When opted-out, a customer no longerreceives any interactive application prompts, or is not able to invokeinteractive applications, even if the CPE is capable of receiving them.

In one embodiment, the following Interactive Opt Out (100) requirementsare supported:

A customer is able to indicate to the interactive application platformof their desire to opt out of receiving or invoking any bound andunbound applications.

When opted-out, the user agent is effectively provisioned with theopt-out setting (assuming default is Opt-in)—and as such the user agentdoes not respond to any interactive application triggers and/or receiveany interactive streamed content.

The customer may also have the option to opt out of certain types ofapplications, in addition to being opted-out of all applications.

SDA Interactive Application Measurement Data—

Overview

Two types of data sets are collected within the SDA 203 relating tointeractive applications: reference data and event data.

In general, the SDA 203 measurement solution relates to the collectionof data directly relating to customer activity with applications, thento enrich this data with information about the application in order tocorrelate usage activities to application functions. The followingsections describe the sets of data to be collected, and the correlationactivities to create the metrics used to support business needs.

Interactive Application Reference Data

Exemplary attributes in Appendix EM of previously incorporated U.S.Provisional Patent Application Ser. No. 61/260,346 describe thereference data required to describe an interactive application.

Interactive Application Event Data

The diagram illustrated in FIG. 30 shows the various interactiveapplication states and transitions between these states which lead tothe generation of interactive application events (stored in SDA-AM203).

Exemplary potential state transitions and the related actions that maybe taken are described in the table of Appendix EN of previouslyincorporated U.S. Provisional Patent Application Ser. No. 61/260,346.

An Interactive Application event may be classified as one of thefollowing types as described in Appendix EM of previously incorporatedU.S. Provisional Patent Application Ser. No. 61/260,346:

-   -   Application Initiation    -   User Prompt    -   User Invocation via Prompt    -   User Invocation via EPG    -   Prompt Time Out    -   Application Utilization    -   User Interaction    -   Application Exit    -   Application Reset    -   Auto-Application    -   Application removal

An “application initiation” interactive application event is generatedwhen a particular interactive application begins spooling to anyparticular set of CPE and or published to an EPG.

A “User Prompt” interactive application event is generated when aninteractive application is invoked by an SCTE trigger, and a prompt isdisplayed on a CPE (or device associated therewith).

A “User Invocation via Prompt” interactive application event isgenerated when a customer invokes an interactive application defined bythe prompt.

A “User Invocation via EPG” interactive application event is generatedwhen a customer selects an EPG menu item invoking a specific interactiveapplication.

A “Prompt Time Out” interactive application event is generated by aninteractive application for each instance where a prompt is not acted onby the user and is removed from the screen when the corresponding boundapplication is no longer presented.

An “Application Utilization” interactive application event is generatedfor each action representing a browse event by a user using anapplication.

A “User Interaction” interactive application event is generated eachtime a user submits a specific request to a back end server while usingan interactive event.

An “Application Exit” interactive application event is generated when acurrently running interactive application is closed due to inactivitytime out, bound programming expiry, application error, and/or specificuser command to exit the application.

An “Application Reset” interactive application event is generated when aCPE removes a previously running application from the run-timeenvironment, as a specific result of an “Application Exit” event.

An “Auto-Application” interactive application event is generated when aparticular application is automatically displayed on a user's CPE (ordisplay device associated therewith) without the need for userinvocation activities.

An “Application removal” event is generated when a particularinteractive application stops being spooled to any particular set of CPEand or un-published on an EPG.

The data of Appendix EO of previously incorporated U.S. ProvisionalPatent Application Ser. No. 61/260,346 may be stored in SDA 203 for eachinteractive application event.

Use Case Examples

The diagram of Appendix EP of previously incorporated U.S. ProvisionalPatent Application Ser. No. 61/260,346 provides examples of how businessuse cases may be supported when storing tuning events as defined herein.In Appendix EQ thereof, one example of the Button/Screen reference isdefined as part of the reference data relating to the specificinteractive applications being measured.

Generation of Interactive Application Business Metrics

Metrics relating to viewership of interactive applications may, in oneembodiment, be generated using the SDA 203 by correlating the referencedata relating to customer QUID, against the tuning, advertisement and/orVOD events which are also stored.

The SDA-AM 203 stores four types of measurement data relating tocustomer usage:

-   -   Tuning Events: which CPE were tuned to which channels in a        particular region and at what time    -   Advertisement Events: which advertisements were played out on a        particular channel, and the region in which that advertisement        was presented    -   VOD Events: which customers viewed which VOD asset and when    -   Interactive application events: which CPE were triggered (either        in a program stream or through subscriber action) to display an        interactive application, and details and timing around        successful/failed display and usage of that interactive        application.

From these four measurement events, a number of business use cases canbe determined by cross correlation of the events by STB GUIDs, CustomerGUIDs, channel and/or date and time.

An example of this follows:

-   -   Tuning events inform the business who was watching a particular        program in a particular region    -   Advertisement events inform if an advertisement was played out,        in that particular region, at the time being investigated    -   Interactive events inform if the programming or advertisement        was enabled with interactive content, and how subscribers        interacted with/ignored the application    -   It is also possible, using tuning events, to determine if the        customer behavior changes from previous viewing habits due to        the presence/use of the interactive application        XLII. Tuning Events—        Introduction—

Measuring the audience for television channels at any particular timerequires collection and analysis of tuning events. Tuning events aregenerated when a customer's CPE is connected to a particular televisionchannel for a particular period of time.

In order to properly identify a valid tuning event, a series ofdefinitions are needed to describe which CPE messages relate to tune-inand tune-out conditions which should be used to govern how tuning eventsare created and subsequently stored in the SDA 203 data stores. As therange of possible events generated from the CPE increases (as is thecase with the ODN and MDN platforms), it is also important to defineevents which do not impact the generation of valid tuning events.

Tuning events are defined herein as being a qualified view by acustomer's CPE of a particular television channel/program on their MAINCPE output, or while viewing the television channel/program, using thePIP function. A qualified view in this case is defined as one in whichthere is a specific consecutive viewing duration on a single Channel,where the CPE tuner is not placed in standby, or deemed as being in azombie or non-responding status. This data can be used to defineimpressions on a particular channel, program or advertisement whichprovides a specific industry metric around viewership data.

Tuning conditions relating to background events, such as second tunerrecordings (Background Recording tunes), and/or recording events basedon a PIP viewing shall also be generated and stored in the SDA

Document Objectives—

The following definitions may be used in obtaining tuning events:

-   -   Definition of valid tuning events stored within the SDA 203 data        store and subsequently used to create business metrics.    -   Definition of CPE events which are qualified as either a valid        start and/or a valid end time of a particular tuning event.    -   Description of abnormal event conditions resulting from        operational errors, which require additional processing to        identify them and improve on accuracy of any resulting metrics.    -   Description of the events which impact the status of a viewing        event but do not qualify as either a start or stop event.        STB Tuner State Definition—

Each CPE comprises at least one tuner. In the case of CPE running theODN middleware platform, a number of virtual tuners are also present.Each tuner can exist in a number of states, and where multiple tunersexist, more than one tuning event may be registered by each CPE. Thetable of Appendix EQ of previously incorporated U.S. Provisional PatentApplication Ser. No. 61/260,346 further illustrates represents exemplaryones of the various states that a tuner can exist in.

A tuning event is generated for a CPE on a tuner basis, in oneembodiment, only if a valid event has been received indicating start ofa tune event (i.e. registering a tune-in status) and an appropriateevent has been received for the same tuner indicating a valid end of atune event (i.e. registering a tune-out status).

Tuning Event Definition—

A tuning event starts when a CPE tuner (or other similar device)successfully completes a tune action and begins to stream a channel onthe main CPE output. In one embodiment, it is only when the MAIN tuneron a CPE is tuned to the same channel for one or more seconds withoutinterruption that a tuning event is considered to have started.

A tuning event ends when a CPE tuner successfully completes a tuneaction onto a different channel than the one which is currentlystreaming on the main CPE output.

A PIP tuning event starts when a CPE tuner successfully completes a tuneaction and begins to stream a channel as a PIP on the main CPE output.In one embodiment, only when the PIP tuner on a CPE is tuned to the samechannel for one or more seconds without interruption is a PIP tuningevent considered to have started.

A PIP tuning event ends when a CPE tuner successfully completes a tuneaction onto a different channel than the one which is currentlystreaming as a PIP on the main CPE output.

A background tuning event starts when a CPE tuner successfully completesa tune action and begins to stream a channel on the background tuner(i.e. not on Main or PIP outputs). In one embodiment, it is only whenthe background tuner on a CPE is tuned to the same channel for one ormore seconds without interruption that a background tuning event isconsidered to have been started.

A background tuning event ends when a CPE tuner successfully completes atune action onto a different channel than the one which is currentlystreaming on the background CPE output.

A tuning event and/or PIP tuning event and/or background tuning eventcan also end on a particular tuner when a CPE tuner enters a forcedzombie state (i.e. when no events are received on an actively tuned CPEtuner within a configurable amount of time (for example, 4 hours oftuner inactivity). The tuning event is generated with the tune out timeafter the configurable amount of time after the tune-in time, and markedas a forced zombie tune-out event.

A tuning event and/or PIP tuning event and/or background tuning eventends when a CPE enters a standby state (i.e. an event is received thatindicates the CPE is actively turned off—e.g. Inactive). In oneembodiment, a tuning event is generated with the tune out time equal tothe inactive event time only when the tune duration equals one or moreseconds without interruption.

A tuning event and/or PIP tuning event and/or background tuning eventends on a particular tuner when a CPE tuner enters a non-responder state(i.e. if the SDV server is unable to reach the CPE, than a non-responderevent is generated). The tuning event is generated with the tune outtime equal to the inactive event time, and marked as an inactivetune-out event. (See the Audit Tracking section for additional details),in one embodiment, only when the tune duration equals one or moreseconds without interruption.

A tuning event and/or PIP tuning event and/or background tuning eventends on a particular tuner when a CPE tuner enters a Measured Zombiestate (i.e. when the SDV server does not receive a response to anykeep-alive messages from the CPE). The tuning event is generated, in oneembodiment, with the tune out time equal to the Aged or Slated eventtime, and marked as a Measured Zombie tune-out event, only when the Tuneduration equals one or more seconds without interruption.

A tuning event and/or PIP tuning event and/or background tuning eventends on a particular tuner when an INIT message is received for thatparticular CPE (i.e. when the SDV server receives an INIT command fromthe CPE when it was in a TUNE_IN state). The tuning event is, in oneembodiment, generated with the tune out time equal to the INIT eventtime, and marked as Error tune-out event, only when the tune durationequals one or more seconds without interruption.

A tuning event and/or PIP tuning event and/or background tuning eventends on a particular tuner when an SDV error is received for thatparticular tuner (i.e. when the SDV server records a specific error forthat tuner). In one embodiment, the tuning event is generated with thetune out time equal to the error event time, and marked as an SDV Errortune-out event, only when the Tune duration equals one or more secondswithout interruption.

In one embodiment, it is possible for a single CPE to register twoconcurrent tuning events if the box is in both MAIN and PIP mode, or inMAIN and background mode at the same time.

Impressions which describe the number of viewers of a particular channel(and which may be of configurable duration) may be measured as thenumber of valid tuning events (i.e. which are in excess of theimpression duration) measured on that channel, (i.e. during the timeinterval in which the impressions are requested), and be furtherfiltered for Main, PIP or background types.

A tuning event does not contain any Personally Identifiable Information(PII) that can be used to associate a tuning event to a customer. Thisincludes the anonymization of both the MAC address of the CPE whichgenerated the tuning event and the customer billing identifierassociated with the MAC address. The algorithm used to anonymize PIIinformation is described elsewhere herein.

The tuning event stored in the SDA-AM 203 data store may consist of thefields such as those illustrated in Appendix ER of previouslyincorporated U.S. Provisional Patent Application Ser. No. 61/260,346(for all valid tuning events over 1 second in duration). In the table ofAppendix ER the “TUNE_OUT_TYPE” values may comprise those listed in thetable of Appendix ES of previously incorporated U.S. Provisional PatentApplication Ser. No. 61/260,346.

Standby Status Definition—

Each CPE tuner is considered as being in standby status when it an eventis received that provides an indication that it no longer requests achannel from the SDV Manager (i.e., only if the CPE has been turned offand an inactive state event is received.

When a standby status is reached, a tuning event (for each CPE tunercurrently in a tune-in status) is generated for the period of timebetween the last valid channel request and the request to enter standbymode (from a tune request to the time the CPE was placed into standbymode).

Zombie Status Definition—

A CPE is in Zombie status when it provides a Last User Activity (LUA)message to the SDV Manager which exceeds the configurable amount bywhich the SDV Manager is required to provide an active stream to the CPE(for example, the CPE sends a LUA of 60 minutes to the SDV Manager, andthe SDV Manager is configured to reclaim a stream if needed after 60minutes).

In one embodiment, for example, in the event that a LUA message is notreceived by the CPE after 60 minutes of continuous viewing of a singlechannel, and a CPE tuner is in a valid tune-in status, a tune-out eventis forced for that tuner. Hence a CPE tuner can enter a Zombie state if(Z1) a message is provided that the CPE tuner LUA exceeds 4 Hours, OR(as in the case that a LUA is not received), the tune-in state for atuner exceeds 4 hours, in which case a Zombie state is forced.

When a Zombie status is reached, a tuning event is generated for eachactive tuner for the period of time between the last valid channelrequest and the LUA message (or the forced tune-out status) whichindicates the reclamation threshold has been reached (i.e.: from thelast valid request for a channel and the LUA message which indicates noactivity from the CPE which exceeds the allowable limit for an activeCPE for the specified SDV Manager).

If a CPE tuner is deemed to be in Zombie state, then any new tunemessages coming from that CPE are treated as new tuning events.

Record Status Definition—

A record event is captured if a channel being actively watched starts torecord the program in either MAIN or PIP mode (for example: if the tuneris not recording the source and then actively/passively begins to recordthe source while tuned).

A Record Event is not collected without an associated tune event. A tuneevent indicates (via a record flag) that the source was being recordedduring the tune.

If a tuner is recording but the source is not actively being watched,the tuning event indicates (via background flag) that the source wastuned to in background mode.

A record event is indicated by setting the record flag on the associatedtuning event to TRUE.

Pay Per View (PPV) Status Definition—

If a tuning event is for a PPV source, the tuning event indicates thatthe source was PPV by setting a flag on the tuning event.

CPE timers which are used to access PPV events (prior to PPV purchase)generate a regular tuning event which describes that the channel beingviewed was the PPV barker channel (i.e. no PPV flag is set), in oneembodiment if, the duration of the event was equal or greater than 1second.

Once a PPV purchase is completed and the PPV event is being watchedanother tuning event is logged to indicate that the CPE tuner waswatching a PPV event (i.e. a PPV flag will be set), in one embodiment,only if the duration of the event was equal or greater than 1 second.

Is the PPV event is being watched in PIP mode, then the tuning event isalso marked as PIP, and the IS_PIP flag shall be marked as “TRUE”

Audit Tracking—

A number of operational conditions, resulting from the day-to-day usageof the above described service, are addressed by the tuning eventdefinition in order to provide additional accuracy of tuning metrics asa result of such events. Appendix ET of previously incorporated U.S.Provisional Patent Application Ser. No. 61/260,346 illustrates exemplaryoperational conditions.

Tuning Event Definitions—

Given the various CPE tuner states, it is contingent on the mediationplatform to capture, analyze and use the states to generate valid tuningevents and/or discard tuner events. The diagram of FIG. 31 illustratesthe various tuner states and a visualization of the valid transitionsbetween these states which would lead to the generation of TUNE IN orOUT events to be used in determining what data is to be stored in SDA-AM203.

Generally, each valid TUNE OUT event generates the appropriate tuningevent as described in the state transition matrix of FIG. 32.

In one embodiment, a tuning event is generated if and only if theduration between the registered tune-in and registered tune-out statusis greater than or equal to 1 second.

A Forced Zombie may occur, in one variant, when a tune-in durationequals 240 minutes.

In another embodiment, a Measured Zombie tune-out occurs if and only ifthe duration between the Registered Tune In and the Measured Zombieevents (i.e. Slated and/or Aged events) is greater than or equal to 1second

An Error tune-out event occurs when an INIT event is received for a STBwhile being in a valid tune-in state and the duration is greater than orequal to 1 second, in one embodiment.

In yet another embodiment, if a channel has been tuned into for at least1 second (i.e. on either PIP, MAIN, or Background mode), and a recordingis initiated on that channel, a tuning event is generated for thenon-recorded part of the viewing, and another tuning event is generated,which contains the IS_REC flag set to TRUE.

If a channel has been tuned into and is recording for at least 1 second(i.e. on either PIP, MAIN, or Background mode), and the recording isstopped, in one embodiment, a tuning event is generated for the recordedpart of the viewing, and another tuning event is generated, whichcontains the IS_REC flag set to FALSE.

Business Use Case Examples—

Examples of how various business use cases may be supported when storingtuning events as defined herein is provided in the diagram of Appendixof EU of previously incorporated U.S. Provisional Patent ApplicationSer. No. 61/260,346.

Anonymous Household—

The first element required for creating an anonymous household profileis the core household information. In one embodiment, the followingalgorithm is applied to arrive at the anonymous household view:

-   -   1) For each billing system within MSO footprint        -   a. For each customer in the billing system            -   i. Collect each household record, including the elements                specified in the data dictionary                -   1. Also include each CPE in the household, with the                    CPE type capabilities infoiniation as specified in                    the data dictionary            -   ii. Collect the Division ID (DI) in the billing system                associated with this customer            -   iii. Collect the Customer ID (CI) for this customer            -   iv. Concatenate DI to the end CI, yielding UID            -   v. Break the UID into as many DWORDs as necessary,                padding the least significant DWORD with zeros            -   vi. From the first DWORD, add the third salt value.                Carry any overflow to the adjacent DWORD.            -   vii. Subtract from the second DWORD fourth salt value.            -   viii. Continue this process, adding the first value to                odd DWORDs, subtracting the second value from even                DWORDs            -   ix. Convert the series of DWORDs back into a string            -   x. Perform an MD5 hash on the resultant string            -   xi. The output of this MD5 hash is the GUID for this                household                Anonymous CPE Association—

Once an anonymous household record has been arrived at, each CPE in thehousehold is associated to that anonymous record, again, in an anonymousway. This is achieved, in one embodiment, through the following process:

-   -   1) For each anonymous household record, a list of CPE associated        with that household is pulled        -   a. For each CPE            -   i. Take each MAC address            -   ii. Break the MAC address into two DWORDs, padding the                second with zeros (e.g. 00:EE:EE:EE:EE would be:                0x00000000 and 0xEEEEEEEE)            -   iii. To the first DWORD, add a first salt value. Carry                any overflow to the second DWORD (e.g. the first DWORD                (e.g., 0xEEEEEEEE) would be: 0xEEEEEEEE+first salt                value=result, however, if this result has “overflowed”                the DWORD limit, a carryover byte is created e.g.,                0x00000001 and the DWORD is limited to 32 bits (e.g., 32                bits of result)            -   iv. From the second DWORD, subtract a second salt value                (e.g. in the previous example, since the second DWORD is                now 0x00000001, thus 0x00000001−second salt                value=result2)            -   v. Re-assemble the resultant DWORDs into a String (e.g.                extending the example we have been building: concatenate                (result2:result))            -   vi. Apply an MD5 hash to the String            -   vii. The resultant HMAC is the GUID for that MAC Address                Anonymous Tuning Records from CPE—

When tuning records are received in the collection system 200, thosetuning records are made anonymous in such a way as to associate them totuning records. In one embodiment, this is accomplished via thefollowing algorithm:

-   -   1) When the tuning record is received, collect (Note: All this        data, along with the tuning record itself MAY NOT be written to        any form of persistent media until anonymization is complete)        -   a. MAC Address from the tuning record        -   b. Division ID from the billing system        -   c. Query the billing system to determine the Customer ID for            the customer associated with the MAC address    -   2) To the Division ID and Customer ID, apply the anonymization        algorithm presented herein “Anonymous Household” (Household        GUID)    -   3) To the MAC Address, apply the anonymization algorithm        presented herein “Anonymous CPE Association” (CPE GUID)    -   4) Remove from the tuning record the MAC address    -   5) Add to the record the Household GUID and the CPE GUID

It will be recognized that while certain aspects of the invention aredescribed in terms of a specific sequence of steps of a method, thesedescriptions are only illustrative of the broader methods of theinvention, and may be modified as required by the particularapplication. Certain steps may be rendered unnecessary or optional undercertain circumstances. Additionally, certain steps or functionality maybe added to the disclosed embodiments, or the order of performance oftwo or more steps permuted. All such variations are considered to beencompassed within the invention disclosed and claimed herein.

While the above detailed description has shown, described, and pointedout novel features of the invention as applied to various embodiments,it will be understood that various omissions, substitutions, and changesin the form and details of the device or process illustrated may be madeby those skilled in the art without departing from the invention. Thisdescription is in no way meant to be limiting, but rather should betaken as illustrative of the general principles of the invention. Thescope of the invention should be determined with reference to theclaims.

What is claimed is:
 1. A method for collecting and analyzing of datarelating to delivery of digital content via a digital network, saidmethod comprising: collecting a plurality of user activity data from oneor more user devices in data communication with said digital network,said user activity data comprising data relating to usage or interactionby a user with said digital content via said one or more user devices,said plurality of user activity data collected via a collection entityin data communication with said network, each of said plurality of useractivity data comprising at least a tune-in time for a user session;receiving a plurality of digital reference data at said collectionentity from a database in data communication with at least saidcollection entity, said digital reference data describing said digitalcontent; identifying one or more of said plurality of user activity datawhich lack a tune-out time corresponding to said tune-in time for saiduser session; in response to said identification, excluding saididentified one or more of said plurality of user activity data whichlack said tune-out time; correlating, via mapping logic stored on astorage device in data communication with said collection entity, atleast a remaining portion of said user activity data to said referencedata to create a plurality of viewing impressions, each of saidremaining portion of said plurality of user activity data comprisingsaid tune-out time corresponding to said tune-in time for said usersession; and transmitting said plurality of viewing impressions fromsaid collection entity to an analysis entity for analysis thereof, saidanalysis entity in data communication with at least said collectionentity.
 2. The method of claim 1, wherein said plurality of useractivity data comprises tuning event data.
 3. The method of claim 1,further comprising: validating at least a portion of said plurality ofuser activity data; anonymizing at least a portion of said plurality ofuser activity data; and enriching at least a portion of said pluralityof user activity data.
 4. The method of claim 3, wherein said act ofvalidating at least a portion of said data comprises ensuring a MediaAccess Control (MAC) address of a device associated with said pluralityof user activity data comprises a MAC address of one of said one or moredevices associated with said user.
 5. The method of claim 4, whereinsaid act of anonymizing at least said portion of said plurality of useractivity data comprises utilizing at least one obfuscation process toobfuscate said MAC address.
 6. The method of claim 3, wherein said actof enriching at least said portion of said plurality of user activitydata comprises adding non-identifiable user-specific information to saidplurality of user activity data.
 7. The method of claim 1, wherein saidanalysis entity comprises a third party entity.
 8. The method of claim1, wherein said network comprises a managed network, and said analysisentity comprises a billing entity within said managed network.
 9. Themethod of claim 1, wherein each of said plurality of user activity datafurther comprises information regarding one or more of: a programviewed; a start/end date/time of said program; a tune-in event type; atune-out event type; and an anonymized Media Access Control (MAC)address.
 10. The method of claim 1, wherein said act of identifying saidone or more of said plurality of user activity data which lack saidtune-out time corresponding to said tune-in time for said user sessioncomprises: calculating a duration of user inactivity after said starttime; and determining said duration of user inactivity is equal to orgreater than a pre-determined threshold.
 11. The method of claim 10,wherein said act of identifying said one or more of said plurality ofuser activity data which lack said tune-out time corresponding to saidtune-in time for said user session further comprises: in response todetermining said duration of user activity is equal to or greater thansaid pre-determined threshold, generating a forced tune-out timecorresponding to said tune-in time for said user session; and assigningan indicator of said forced tune-out time to each of said identified oneor more of said plurality of user activity data which lack said tune-outtime.
 12. A method for managing digital data in an operator-manageddigital content delivery network, said method comprising: receiving afirst plurality data at a collection entity in data communication withsaid managed digital content delivery network, said first plurality ofdata comprising a plurality of fields comprising data relating to asubscriber's interaction with a plurality of content delivered theretofrom at least one user device in data communication with said manageddigital content delivery network; determining validity of at least aportion of said first plurality of data so as to ensure said at leastone user device from which said first plurality of data is received isassociated with said subscriber; in response to determining validity ofsaid at least portion of said first plurality of data, parsing saidvalidated portion of said first plurality of data to individual ones ofsaid plurality of fields; receiving a second plurality of data from adatabase in data communication with said managed digital contentdelivery network, said second plurality of data comprising referencedata describing said plurality of content; and processing saidindividual ones of said plurality of fields to generate a plurality ofdigital records correlating said individual ones of said plurality offields to said reference data, said plurality of records correlatingsaid individual ones of said plurality of fields to said reference databeing utilized to describe a period of time in which said subscriber ispresumed to be utilizing a particular one of said plurality of contentat said at least one user device.
 13. The method of claim 12, furthercomprising transmitting said plurality of records correlating saidindividual ones of said plurality of fields to said reference data to acomputerized analysis entity configured to utilize information containedtherein to derive viewing patterns.
 14. The method of claim 12, furthercomprising transmitting said plurality of records correlating saidindividual ones of said plurality of fields to said reference data to ananalysis entity of said managed network configured to create at leastone report describing behavior of said subscriber or a plurality ofsubscribers.
 15. The method of claim 14, wherein said at least onereport comprises a report of viewership of primary content containedwithin said particular one of said plurality of content for a pluralityof user devices as a function of time, said plurality of user deviceseach associated with at least said subscriber, said at least one userdevice being one of said plurality of user devices.
 16. The method ofclaim 14, wherein said at least one report comprises a report ofviewership of secondary content contained within said particular one ofsaid plurality of content for a plurality of user devices as a functionof time, said plurality of user devices each associated with at leastsaid subscriber, said at least one user device being one of saidplurality of user devices.
 17. The method of claim 14, wherein said atleast one report comprises a report of video on demand (VOD) usage for aplurality of user devices, said plurality of user devices eachassociated with at least said subscriber, said at least one user devicebeing one of said plurality of user devices.
 18. The method of claim 14,wherein said at least one report comprises a report of trick modeoperation of said at least one user device during playback of at leastone of: (i) primary content contained within said particular one of saidplurality of content, and/or (ii) secondary content contained withinsaid particular one of said plurality of content.
 19. The method ofclaim 12, further comprising implementing at least one error recoverymechanism in an instance where at least one other portion of said firstplurality of data is determined to be not valid.
 20. Apparatus forcollection and analysis of data relating to delivery of content in anetwork, said apparatus comprising at least one collection entitycomprising: a first processor; and a first non-transitory computerreadable storage medium having a first computer program stored thereon,said first computer program configured to, when executed, cause saidprocessor to: collect a first plurality of data relating to a pluralityof primary and secondary digital content delivered to a plurality ofsubscriber devices via said network; and collect a second plurality ofdata relating to individual ones of a plurality of tuning events whichoccur at said plurality of subscriber devices, said plurality ofsubscriber devices each being associated with one or more of a pluralityof subscribers, said second plurality of data comprising a plurality offields; determine validity of said second plurality of data; in responseto validation of at least a portion of said second plurality of data,parse said validated portion of said second plurality of data intoindividual ones of said plurality of fields; and in response toinvalidation of at least one other portion of said plurality of datarelating to said individual ones of said plurality of tuning events,store said invalidated portion of said plurality of data relating tosaid individual ones of said plurality of tuning events in an errortable.
 21. The apparatus of claim 20, further comprising at least oneanalysis entity in data communication with said at least one collectionentity, said at least one analysis entity comprising: a secondprocessor; and a second non-transitory computer readable storage mediumhaving a second computer program stored thereon, said second computerprogram configured to, when executed, cause said second processor to:utilize said first plurality of data and said second plurality of datato correlate said plurality of tuning events to individual ones of saidplurality of primary and secondary content; and generate one or morereports configured to illustrate behaviors of said plurality ofsubscriber devices with respect to said primary and secondary content.22. The apparatus of claim 20, wherein said first computer program isfurther configured to, when executed, cause said processor to: anonymizesaid second plurality of data in order to obfuscate an identity of saidplurality of subscriber devices from which said data is received. 23.Network apparatus for use in a network, comprising: an interfaceconfigured to communicate with said network; processing apparatus incommunication with said interface; and at least one storage device incommunication with said processing apparatus, said storage devicecomprising a computer program which is configured to, when executed,cause said processing apparatus to: obtain data relating to at least oneof each primary and secondary content elements delivered to a pluralityof client apparatus via said network; collect a plurality of datarelating to individual ones of a plurality of tuning events which occurat said plurality of client apparatus; validate said plurality ofcollected data to identify one or more of a valid portion of saidcollected plurality of data and an invalid portion of said collectedplurality of data; in response to identification of said invalid portionof said collected plurality of data, implement at least one recoverymechanism to recycle said invalid portion of said collected plurality ofdata; and in response to identification of said valid portion of saidcollected plurality of data, process said obtained data and said validportion of said collected plurality of data so as to identify one ormore correlations between said tuning events and said delivered primaryand secondary content elements.